-- Logs begin at Thu 2022-08-11 15:36:37 UTC, end at Thu 2022-08-11 17:14:21 UTC. -- Aug 11 16:14:17.946749 dv-cinder-852208-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 11 16:14:17.955265 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 11 16:14:17 2022] *** Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: nodename: dv-cinder-852208-4-iscsi Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: machine: x86_64 Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: clock source: unix Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: pcre jit disabled Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: detected number of CPU cores: 4 Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: current working directory: / Aug 11 16:14:17.957189 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: detected binary path: /usr/bin/uwsgi-core Aug 11 16:14:17.957720 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: your processes number limit is 47680 Aug 11 16:14:17.957720 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: your memory page size is 4096 bytes Aug 11 16:14:17.957720 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: detected max file descriptor number: 2048 Aug 11 16:14:17.957720 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: lock engine: pthread robust mutexes Aug 11 16:14:17.957720 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: thunder lock: enabled Aug 11 16:14:17.957720 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 11 16:14:17.957720 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 11 16:14:17.996574 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: Python main interpreter initialized at 0x5563914039c0 Aug 11 16:14:17.996574 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: python threads support enabled Aug 11 16:14:17.996574 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: your server socket listen backlog is limited to 100 connections Aug 11 16:14:17.996574 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: your mercy for graceful operations on workers is 90 seconds Aug 11 16:14:17.996944 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: mapped 403077 bytes (393 KB) for 2 cores Aug 11 16:14:17.997023 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: *** Operational MODE: preforking *** Aug 11 16:14:17.997255 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: *** uWSGI is running in multiple interpreter mode *** Aug 11 16:14:17.997255 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: spawned uWSGI master process (pid: 81419) Aug 11 16:14:17.997453 dv-cinder-852208-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 11 16:14:17.998016 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: spawned uWSGI worker 1 (pid: 81427, cores: 1) Aug 11 16:14:17.998347 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: spawned uWSGI worker 2 (pid: 81428, cores: 1) Aug 11 16:14:18.000144 dv-cinder-852208-4-iscsi devstack@keystone.service[81419]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 11 16:14:23.585820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 16:14:23.585820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: from cryptography.utils import int_from_bytes Aug 11 16:14:23.586736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 16:14:23.586736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: from cryptography.utils import int_from_bytes Aug 11 16:14:23.590250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 16:14:23.590250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: from cryptography.utils import int_from_bytes Aug 11 16:14:23.590705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 11 16:14:23.590705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: from cryptography.utils import int_from_bytes Aug 11 16:14:23.637171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 11 16:14:23.637349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 11 16:14:23.637545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] command line args: [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 11 16:14:23.637818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 11 16:14:23.637999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ================================================================================ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 11 16:14:23.638316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] admin_token = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.638634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_dir = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.638870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.639077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] config_source = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.639298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] debug = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.639521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.639778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 11 16:14:23.639927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 11 16:14:23.640007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.640136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] command line args: [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 11 16:14:23.640216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.640421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 11 16:14:23.640499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.640610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ================================================================================ {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 11 16:14:23.640797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.640881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] admin_token = **** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] config_dir = [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_config_append = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] config_source = [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] debug = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_dir = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.641902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.642905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] list_limit = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] log_config_append = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] log_dir = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] log_file = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.643946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.644070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.644257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.644353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.644522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.644608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.644685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.644837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.644962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.645093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.645171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.645340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.645423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.645576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.645724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.645818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.646025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.646112 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.646258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.646504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.646681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.646770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.646882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] publish_errors = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.647010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.647173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.647469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.647550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.647766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] publish_errors = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.647870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.648074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.648556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.648634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.648797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.648898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] standard_threads = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.648982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.649143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.649219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.649448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.649533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.649658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.649785 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] standard_threads = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.649859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_journal = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.649935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_json = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_stderr = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] use_journal = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] use_syslog = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] use_json = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] use_stderr = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.650980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] use_syslog = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.651080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.651212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 11 16:14:23.651382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.651459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.651695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.651771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.651958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.652035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.652224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.652413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.652504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.652707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.652783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.652959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.653067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.653197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.653419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.653497 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.653685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.653877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.653954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.654159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.654240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.654407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.654490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.654646 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.654723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.654894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.654984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.655146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.655228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.655385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.655462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.655642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.655794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.655874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.655945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.656148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.656228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.656389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.656479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.656645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.656724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.656883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.656958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.657142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.657232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.657392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.657474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.657629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.657706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.657862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.657950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.658093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.658170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.658326 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.658401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.658559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.658636 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.658795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.658872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.659983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.660060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.660256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.660333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.660490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.660566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.660729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.660813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.660962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661655 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.661971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.662127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.662204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.662364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.662441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.662599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.662681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.662839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.662916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.663071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.663148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.663359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.663543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.663728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.663972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.664143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.664265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.664409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.664514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.664650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.664761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.664896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.665918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.666066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.666143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.666317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.666396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.666551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.666641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.666787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.666872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.667987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.668064 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.668249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.668327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.668519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.668597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.668759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.668841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.669961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.670037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.670209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.670285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.670447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.670531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.670684 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.670761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.670926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.671951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.672126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.672206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.672386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.672464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.672627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.672719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.672869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.672951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.673106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.673190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.673348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.673500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.673575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.673648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.673809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.673886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.674967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.675051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.675196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.675281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.675436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.675513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.675670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.675756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.675921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.676005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.676182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.676263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.676440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.676528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.676699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.676776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.676939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.677015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.677171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] auth.external = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.677249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.external = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.677401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.677478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.677701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.677818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.677933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.678012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.678165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] auth.password = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.678243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.password = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.678396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] auth.token = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.678472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] auth.token = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.678634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.678713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.678870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.679460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.679671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.679909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.680027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.680188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.680282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.680442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.680542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.680678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.680795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.680917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.681032 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.681169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.681273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] credential.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.681419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.681525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.681670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.681780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.681910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.682030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.682155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.682270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.682391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.682515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.682655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.682753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.682911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.683019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.683148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.683276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.683404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.683516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.683685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.683763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.683932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.684042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.684220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.684312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.684463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.684579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.684721 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.684809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.684959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.685070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.685199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.685307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] federation.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.685448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.685535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.685697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.685776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.685930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.686029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.686179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.686266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.686418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.686497 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.686675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.686760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.686915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.687973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.688152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.688233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.688407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.688561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.688637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.688709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.688869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.688949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.689356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.689436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.689595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.689673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.689830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.689906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.690071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.690151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.690323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.690401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.690556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.690633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.690792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.690871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691099 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.691996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.692074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.692268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.692349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.692515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.692593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.692779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.692856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.693936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.694014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.694173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.694253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.694413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.694491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.695268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.695350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.695549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.695806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.695958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.696064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.696226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.696337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.696484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.696593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.696970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.697933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.698970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.699132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.699211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.699364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.699441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.699598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.699678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.699849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.699926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.700084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.700220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.700341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.700423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.700581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.700658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.700822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.700902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.701054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.701134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.701291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.701368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.701528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.701607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.701763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.701842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.702959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.703968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.704146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.704237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.704387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.704474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.704621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.704700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.704873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.704952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.705123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.705210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.705357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.705435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.705607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.705685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.705846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.705933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.706078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.706155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.706313 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.706390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.706550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.706640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.706789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.706869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.707992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.708079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.708254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.708334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.708492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.708569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.708728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.708818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.708964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.709990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.710182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.710276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.710575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.710654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.710872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] resource.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.711039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.711220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.711309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.711475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.711554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.711718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.711797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.711972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.712059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.712252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.712334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.712494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.712572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.712735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.712896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.712973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] role.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713687 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] role.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.713995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.714168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.714256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.714416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.714495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.714663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.714741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.714910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.714989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.715157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.715235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.715390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.715468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.715639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.715719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.715887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.715965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.716148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.716273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.716401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.716493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.716653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.716795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.716905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.717035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.717156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.717289 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.717542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.717620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.717783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.717898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.718036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.718155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.718289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.718397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.718540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.718791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.718913 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.719028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.719157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.719391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.719469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.719625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.719704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.719865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.719944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.720114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.720207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.720364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.720444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.720602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.720681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.720857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.720945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.721095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.721173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.721338 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.721417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.721578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.721665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.721817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.721895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.722048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.722126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.722286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] token.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.722373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.722526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.722605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.722777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.722856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.723989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.724067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.724254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.724335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.724492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.724583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.724739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.724818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.724980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.725059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.725212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.725297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.725458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.725537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.725709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.725824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.726032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.726351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.726439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.726619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.726699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.726861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.726941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.727107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.727187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.connection = **** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.727345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.727437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.727585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.727664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.727823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.727902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.728062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.728201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.728333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.728414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.728572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.728651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.728815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.728903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.729054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.729133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.729303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.729383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.729538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.729627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.729782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.729861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.730020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.730099 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.730262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.730349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.730499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.730577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.730670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81428) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 11 16:14:23.730748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.730965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.731207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.731444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 11 16:14:23.731616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 11 16:14:23.761181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.761409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.761635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.761928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.762163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.762390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.772233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.778397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.801308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.801542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.805454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.805690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.855723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.856605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.907333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.907569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.909095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.909329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.973068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.973265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.973361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.973497 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.973582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.973713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.973798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.973925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.974017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.974136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.974220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.974351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.997496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.997736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.997833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.997949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.998892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:23.999075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.009968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.010616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.022484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.022746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.022941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.023928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.024015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.024232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81428) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 11 16:14:24.041331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.041582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 16:14:24.041672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.041857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 16:14:24.041942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.042056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 16:14:24.042200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.042326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.042404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 11 16:14:24.042584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.042670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.042847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.042932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.043104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.043187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.043394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.055123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.055402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.055570 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.055661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.056336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.056443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.056528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.056528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.062860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.063281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.066187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.066472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.066643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.066756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.066926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.067034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.067209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.067312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.067490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.067572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.067773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.067863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.068036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.068143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.068329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.068428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.068605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.068711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.068870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.068950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.069140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.069406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.091813 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.092605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.093325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.093608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.094096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.094385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.098191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.098474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.098752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.099031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.099316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.099601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.105930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.106170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.107211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.107505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.110256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.110546 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.110828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.111110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.111397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.111668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.111752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.111942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.112023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.112244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.112329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.112554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.112834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.113126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.113406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.113675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.128739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.129018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.130249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.130534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.133055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.134619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.136202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.136498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.137796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.138097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.140907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.142523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.144069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.145746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.147224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.147522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.147832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.148144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.148428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.148706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.149001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.149283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.149581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.149877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.150160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.150440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.165238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.165533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.165799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.166057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.167293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.167584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.167856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.168147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.173128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.174676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.174953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.175163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.176754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.177037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.177780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.178063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.178358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.178649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.179588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.180689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.180976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.181276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.181565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.181843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.191480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.191761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.192044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.192352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.192635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.192928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.193202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.193609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.193893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.194171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.194449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.194866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.195143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.195425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.209912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.212480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.212904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.215463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.215821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.217407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.218455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.218900 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.219189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.219470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.220039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.221563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.221850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.222143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.226519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.228150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.229317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.230930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.231061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.231340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.231622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.231895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.233896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.234190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.234474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.234747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.239928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.240240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.240520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.243465 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.243752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.244035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.248016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.248325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.248606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.248884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.249159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.249438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.249709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.249989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.250369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.250642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.250915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.251093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81428) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 11 16:14:24.251223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.251380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.251522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.251652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.251783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.251932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.252216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.252494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.252769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.253048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.253430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.253713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.253988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.254254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.254535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.254810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81428) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 11 16:14:24.285203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5563914039c0 pid: 81427 (default app) Aug 11 16:14:24.288973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ea0ad-9ff7-4268-ad1b-f2c252e32aa7 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:24.289268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ea0ad-9ff7-4268-ad1b-f2c252e32aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:24.289525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ea0ad-9ff7-4268-ad1b-f2c252e32aa7 None None] PATH_INFO: `/v3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:24.289658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5563914039c0 pid: 81428 (default app) Aug 11 16:14:24.293790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Thu Aug 11 16:14:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:31.904657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd00d1-9683-4d37-ba47-64648f168dbd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:31.905222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd00d1-9683-4d37-ba47-64648f168dbd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:31.905222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd00d1-9683-4d37-ba47-64648f168dbd None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:31.909609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1/2] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:31.915708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:31.916169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:31.916341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:31.986998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: INFO dbcounter [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] Registered counter for database keystone Aug 11 16:14:32.005944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG oslo_db.sqlalchemy.engines [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81427) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 11 16:14:32.012070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writer thread running {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 11 16:14:32.191221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG passlib.handlers.bcrypt [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81427) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 11 16:14:32.191562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG passlib.handlers.bcrypt [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81427) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 11 16:14:32.246451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:32.248055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:32.377145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 2/3] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 11 16:14:32.512077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd00d1-9683-4d37-ba47-64648f168dbd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:32.579885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: INFO dbcounter [None req-1bdd00d1-9683-4d37-ba47-64648f168dbd None None] Registered counter for database keystone Aug 11 16:14:32.601516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG oslo_db.sqlalchemy.engines [None req-1bdd00d1-9683-4d37-ba47-64648f168dbd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81428) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 11 16:14:32.608029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writer thread running {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 11 16:14:32.724219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd00d1-9683-4d37-ba47-64648f168dbd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:32.728502 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dce2b2-0a6c-4e87-8623-827007ff2542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:32.729115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce2b2-0a6c-4e87-8623-827007ff2542 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:32.729392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce2b2-0a6c-4e87-8623-827007ff2542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:32.729661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dce2b2-0a6c-4e87-8623-827007ff2542 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:32.737567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dce2b2-0a6c-4e87-8623-827007ff2542 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:32.790108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f1dce2b2-0a6c-4e87-8623-827007ff2542 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 11 16:14:32.790394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dce2b2-0a6c-4e87-8623-827007ff2542 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:32.799660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 2/4] 10.222.0.43 () {58 vars in 1324 bytes} [Thu Aug 11 16:14:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 11 16:14:32.806138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:32.821341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da81a6-2876-4119-a4c7-9252defed73c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:32.825775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c53a66-a602-4128-b773-678cf71a83cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:32.826855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53a66-a602-4128-b773-678cf71a83cf admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:32.827133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53a66-a602-4128-b773-678cf71a83cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:32.827402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53a66-a602-4128-b773-678cf71a83cf admin admin] PATH_INFO: `/v3/projects/307850a266084dc1bb810ff53de8b0aa` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:32.828852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c53a66-a602-4128-b773-678cf71a83cf admin admin] RBAC: Authorizing `identity:get_project(project_id=307850a266084dc1bb810ff53de8b0aa)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:32.880493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b1c53a66-a602-4128-b773-678cf71a83cf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 11 16:14:32.880832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c53a66-a602-4128-b773-678cf71a83cf admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:32.885924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 3/5] 10.222.0.43 () {56 vars in 1207 bytes} [Thu Aug 11 16:14:32 2022] GET /identity/v3/projects/307850a266084dc1bb810ff53de8b0aa => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:34.880297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9bf06-5ade-4116-a18f-4b54fff2ee03 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:34.880853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9bf06-5ade-4116-a18f-4b54fff2ee03 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:34.880853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9bf06-5ade-4116-a18f-4b54fff2ee03 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:34.885094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 3/6] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:14:34.890480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022c21-31cb-4744-a96a-b65e86b2d133 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:34.890899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022c21-31cb-4744-a96a-b65e86b2d133 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:34.891041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022c21-31cb-4744-a96a-b65e86b2d133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:34.913525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-71022c21-31cb-4744-a96a-b65e86b2d133 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:34.915063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-71022c21-31cb-4744-a96a-b65e86b2d133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:34.922951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 4/7] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 11 16:14:34.944759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9bf06-5ade-4116-a18f-4b54fff2ee03 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:34.956581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9bf06-5ade-4116-a18f-4b54fff2ee03 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:34.961439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf827b-2ce1-4ff9-bf67-1845ba5e014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:34.962027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf827b-2ce1-4ff9-bf67-1845ba5e014b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:34.962303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf827b-2ce1-4ff9-bf67-1845ba5e014b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:34.962579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf827b-2ce1-4ff9-bf67-1845ba5e014b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:34.970042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf827b-2ce1-4ff9-bf67-1845ba5e014b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:34.971789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf827b-2ce1-4ff9-bf67-1845ba5e014b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:34.981113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 4/8] 10.222.0.43 () {58 vars in 1324 bytes} [Thu Aug 11 16:14:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:14:34.986327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71022c21-31cb-4744-a96a-b65e86b2d133 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:34.995773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71022c21-31cb-4744-a96a-b65e86b2d133 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:35.000439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be290aba-ef44-431d-9e30-b205698dd651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:35.001062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be290aba-ef44-431d-9e30-b205698dd651 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:35.001341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be290aba-ef44-431d-9e30-b205698dd651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:35.001616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be290aba-ef44-431d-9e30-b205698dd651 admin admin] PATH_INFO: `/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:35.002941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be290aba-ef44-431d-9e30-b205698dd651 admin admin] RBAC: Authorizing `identity:get_user(user_id=ddf38f7f3d7248769f04c04a1b75ce6c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:35.004798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be290aba-ef44-431d-9e30-b205698dd651 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:35.006989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 5/9] 10.222.0.43 () {56 vars in 1198 bytes} [Thu Aug 11 16:14:34 2022] GET /identity/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:37.046817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6a2de-c9e7-4ba1-bece-1b3ea8cf833c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.047352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6a2de-c9e7-4ba1-bece-1b3ea8cf833c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.047352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6a2de-c9e7-4ba1-bece-1b3ea8cf833c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.048747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 5/10] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:37.053730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ace52-4d4a-440a-9031-e747b40f989c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.054258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ace52-4d4a-440a-9031-e747b40f989c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.054258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ace52-4d4a-440a-9031-e747b40f989c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.072899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-242ace52-4d4a-440a-9031-e747b40f989c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:37.074075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-242ace52-4d4a-440a-9031-e747b40f989c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:37.153485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 6/11] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:14:37.170685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214b1f7-6e31-4d76-856d-e85e1494b022 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.170971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214b1f7-6e31-4d76-856d-e85e1494b022 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.171249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5214b1f7-6e31-4d76-856d-e85e1494b022 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.172604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 6/12] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:14:37.175845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ace52-4d4a-440a-9031-e747b40f989c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:37.177488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.177776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.178048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.201538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:37.221741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG passlib.handlers.bcrypt [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81428) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 11 16:14:37.222165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG passlib.handlers.bcrypt [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81428) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 11 16:14:37.267955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:37.269566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:37.281412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ace52-4d4a-440a-9031-e747b40f989c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:37.316031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700a69c-7fa2-4461-857d-b230b95006c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:37.316697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700a69c-7fa2-4461-857d-b230b95006c7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.316973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700a69c-7fa2-4461-857d-b230b95006c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.317243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700a69c-7fa2-4461-857d-b230b95006c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.328964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700a69c-7fa2-4461-857d-b230b95006c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-a700a69c-7fa2-4461-857d-b230b95006c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:37.337647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:37.338705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:14:37.339540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 7/13] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:14:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:37.346273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700a69c-7fa2-4461-857d-b230b95006c7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:37.369985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 7/14] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:14:37.392118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:37.427927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:37.447625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700a69c-7fa2-4461-857d-b230b95006c7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:37.481730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:37.482332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.482606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.482874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.483537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:37.483833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:37.485234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:37.492707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac3884-3c4b-4c83-8de0-69042a41a1e6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:37.494985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:37.498816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 8/15] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:37.504340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:37.531997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d271286-c559-454c-a24d-83092b2d0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:37.533109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d271286-c559-454c-a24d-83092b2d0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.533390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d271286-c559-454c-a24d-83092b2d0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.533662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d271286-c559-454c-a24d-83092b2d0ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.606115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d271286-c559-454c-a24d-83092b2d0ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:37.607662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d271286-c559-454c-a24d-83092b2d0ce6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:37.710503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24959e5-5319-4c2d-8053-87a4920d4e27 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:37.712440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 8/16] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 16:14:37.718159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d271286-c559-454c-a24d-83092b2d0ce6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:37.721594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:37.746645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e69875-2888-4f20-9a9b-012fa5a8ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:37.747277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e69875-2888-4f20-9a9b-012fa5a8ac51 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.747570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e69875-2888-4f20-9a9b-012fa5a8ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.747842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e69875-2888-4f20-9a9b-012fa5a8ac51 None admin] PATH_INFO: `/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.749013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e69875-2888-4f20-9a9b-012fa5a8ac51 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf38f7f3d7248769f04c04a1b75ce6c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:37.750474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e69875-2888-4f20-9a9b-012fa5a8ac51 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:37.752045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 9/17] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:37 2022] GET /identity/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:37.757498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e69875-2888-4f20-9a9b-012fa5a8ac51 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:37.771914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:37.817493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d271286-c559-454c-a24d-83092b2d0ce6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:37.851339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:37.852233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.852532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.852798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.858023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e69875-2888-4f20-9a9b-012fa5a8ac51 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:37.871360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:37.872843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:14:37.890832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:37.891743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:14:37.892597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:14:37.892597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:14:37.892597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:14:37.892597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 9/18] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:14:37.893700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9d3cb-7b4a-4d9e-87ad-5a4199f7da36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:37.894341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9d3cb-7b4a-4d9e-87ad-5a4199f7da36 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:37.894622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9d3cb-7b4a-4d9e-87ad-5a4199f7da36 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:37.894894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9d3cb-7b4a-4d9e-87ad-5a4199f7da36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:37.896032 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9d3cb-7b4a-4d9e-87ad-5a4199f7da36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:37.898095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9d3cb-7b4a-4d9e-87ad-5a4199f7da36 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:37.898546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:37.899592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 10/19] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:37.905096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9d3cb-7b4a-4d9e-87ad-5a4199f7da36 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:37.954497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:37.986048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:37.998724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45614-0bc0-4d16-815e-df4436064837 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:38.007042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9d3cb-7b4a-4d9e-87ad-5a4199f7da36 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:38.031884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7baf5-362e-447a-985a-8e518b2daafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:38.032520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7baf5-362e-447a-985a-8e518b2daafd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:38.032805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7baf5-362e-447a-985a-8e518b2daafd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:38.033073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7baf5-362e-447a-985a-8e518b2daafd None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:38.033720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7baf5-362e-447a-985a-8e518b2daafd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:38.034021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7baf5-362e-447a-985a-8e518b2daafd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:38.035400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a7baf5-362e-447a-985a-8e518b2daafd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:38.042148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e33bad-b4ce-4584-b04f-bb6bfa1693ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:38.042757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33bad-b4ce-4584-b04f-bb6bfa1693ed None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:38.043036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33bad-b4ce-4584-b04f-bb6bfa1693ed None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:38.043306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e33bad-b4ce-4584-b04f-bb6bfa1693ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:38.044031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33bad-b4ce-4584-b04f-bb6bfa1693ed None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.domain.id=default, user.id=ddf38f7f3d7248769f04c04a1b75ce6c) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:38.044413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33bad-b4ce-4584-b04f-bb6bfa1693ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:38.045847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e33bad-b4ce-4584-b04f-bb6bfa1693ed None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:38.064031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 11/20] 10.222.0.43 () {56 vars in 1322 bytes} [Thu Aug 11 16:14:37 2022] GET /identity/v3/role_assignments?user.id=ddf38f7f3d7248769f04c04a1b75ce6c&scope.domain.id=default&role.id=29fd031edd2b4f65aed8942853150f92 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:38.065261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 10/21] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:40.369695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a9600-3a07-4b3a-b83b-f711565ea367 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:40.370689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a9600-3a07-4b3a-b83b-f711565ea367 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:40.371172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a9600-3a07-4b3a-b83b-f711565ea367 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:40.372974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 12/22] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:40.385921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d204-5af7-495f-95c0-0c3311099e22 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:40.386821 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d204-5af7-495f-95c0-0c3311099e22 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:40.387303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d204-5af7-495f-95c0-0c3311099e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:40.417116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4321d204-5af7-495f-95c0-0c3311099e22 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:40.418670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4321d204-5af7-495f-95c0-0c3311099e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:40.478749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c23dc-4295-4a39-b2e4-046ba21a2f3e None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:40.479059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c23dc-4295-4a39-b2e4-046ba21a2f3e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:40.479292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49c23dc-4295-4a39-b2e4-046ba21a2f3e None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:40.480675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 13/23] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:40.485454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc147-ed6d-4427-bf5f-c75e325d6ad0 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:40.485738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc147-ed6d-4427-bf5f-c75e325d6ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:40.486013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc147-ed6d-4427-bf5f-c75e325d6ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:40.504917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-162bc147-ed6d-4427-bf5f-c75e325d6ad0 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:40.506978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-162bc147-ed6d-4427-bf5f-c75e325d6ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:40.509655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 11/24] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:40.542842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321d204-5af7-495f-95c0-0c3311099e22 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:40.612577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 14/25] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:14:40.613844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3092ad-b3a0-407d-b424-2023cdeaaaf4 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:40.614130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3092ad-b3a0-407d-b424-2023cdeaaaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:40.614401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3092ad-b3a0-407d-b424-2023cdeaaaf4 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:40.621106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 15/26] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:40.627743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1f0c-921b-415b-9926-a054cc548854 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:40.628022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1f0c-921b-415b-9926-a054cc548854 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:40.628315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd1f0c-921b-415b-9926-a054cc548854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:40.650213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:40.663824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e8dd1f0c-921b-415b-9926-a054cc548854 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:40.665053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e8dd1f0c-921b-415b-9926-a054cc548854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:40.718536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321d204-5af7-495f-95c0-0c3311099e22 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:40.782714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 16/27] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:40.783662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd1f0c-921b-415b-9926-a054cc548854 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:40.788487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:40.900824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cd878-9216-421e-852a-4e191293dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:40.901398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd878-9216-421e-852a-4e191293dc8a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:40.901720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd878-9216-421e-852a-4e191293dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:40.901992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cd878-9216-421e-852a-4e191293dc8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:40.929719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2cd878-9216-421e-852a-4e191293dc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ee2cd878-9216-421e-852a-4e191293dc8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:40.940370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:40.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:14:40.941988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 12/28] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:14:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:14:40.945535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cd878-9216-421e-852a-4e191293dc8a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:40.948661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd1f0c-921b-415b-9926-a054cc548854 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:40.988651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480463c-4b81-4dce-a309-44f32da342cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:40.989365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480463c-4b81-4dce-a309-44f32da342cb None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:40.989650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480463c-4b81-4dce-a309-44f32da342cb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:40.989927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480463c-4b81-4dce-a309-44f32da342cb None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:40.991154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480463c-4b81-4dce-a309-44f32da342cb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:40.992659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480463c-4b81-4dce-a309-44f32da342cb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:41.142560 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 17/29] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Aug 11 16:14:40 2022] POST /identity/v3/roles => generated 219 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:14:41.145071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480463c-4b81-4dce-a309-44f32da342cb None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:41.164906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cd878-9216-421e-852a-4e191293dc8a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:41.176704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:41.257065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75b6c-35e1-4bfb-9414-60a8930bb75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:41.257744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75b6c-35e1-4bfb-9414-60a8930bb75e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.258150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75b6c-35e1-4bfb-9414-60a8930bb75e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.258443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75b6c-35e1-4bfb-9414-60a8930bb75e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.278718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a75b6c-35e1-4bfb-9414-60a8930bb75e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:41.281679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a75b6c-35e1-4bfb-9414-60a8930bb75e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-08a75b6c-35e1-4bfb-9414-60a8930bb75e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:14:41.309628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:41.310776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:14:41.311803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:14:41.311803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:14:41.311803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:14:41.313411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 13/30] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:41.320580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:41.321933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0839c1-64e0-42c7-85ac-a57091a4c9a3 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.322431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0839c1-64e0-42c7-85ac-a57091a4c9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.322883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0839c1-64e0-42c7-85ac-a57091a4c9a3 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.325168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 14/31] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:41.326456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0839c1-64e0-42c7-85ac-a57091a4c9a3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:41.370114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480463c-4b81-4dce-a309-44f32da342cb None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:41.415912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a48108-3e48-4bfc-9f43-f953dc1bc1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:41.416952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48108-3e48-4bfc-9f43-f953dc1bc1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.417405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48108-3e48-4bfc-9f43-f953dc1bc1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.418337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a48108-3e48-4bfc-9f43-f953dc1bc1a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.419329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a48108-3e48-4bfc-9f43-f953dc1bc1a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:41.419887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a48108-3e48-4bfc-9f43-f953dc1bc1a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:41.421742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a48108-3e48-4bfc-9f43-f953dc1bc1a7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:41.434282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 18/32] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:41.435956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e237a-5f05-4640-9423-0b64156a4548 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.440558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e237a-5f05-4640-9423-0b64156a4548 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.440768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e237a-5f05-4640-9423-0b64156a4548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.462671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-b54e237a-5f05-4640-9423-0b64156a4548 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:41.464226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b54e237a-5f05-4640-9423-0b64156a4548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:41.467181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0839c1-64e0-42c7-85ac-a57091a4c9a3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:41.518653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:41.519682 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.520449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.521452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.523439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:41.523971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:41.526254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:41.542284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 15/33] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:41.543506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:41.546114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 19/34] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:41.553269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e237a-5f05-4640-9423-0b64156a4548 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:41.567275 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:41.632876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:41.684608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53fb79-149d-44e0-b448-3bb204a2e0f6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:41.686943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e237a-5f05-4640-9423-0b64156a4548 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:41.724258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d71e0-58f6-4620-afa2-d96bae762153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:41.725300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d71e0-58f6-4620-afa2-d96bae762153 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.725693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d71e0-58f6-4620-afa2-d96bae762153 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.726087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d71e0-58f6-4620-afa2-d96bae762153 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.726829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d71e0-58f6-4620-afa2-d96bae762153 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:41.728510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d71e0-58f6-4620-afa2-d96bae762153 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:41.729869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b6c98-bf89-4e54-b19a-520c16fcbea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:41.730662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b6c98-bf89-4e54-b19a-520c16fcbea3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.731085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b6c98-bf89-4e54-b19a-520c16fcbea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.731517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b6c98-bf89-4e54-b19a-520c16fcbea3 None admin] PATH_INFO: `/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.733185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b6c98-bf89-4e54-b19a-520c16fcbea3 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf38f7f3d7248769f04c04a1b75ce6c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:41.735505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b6c98-bf89-4e54-b19a-520c16fcbea3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:41.737290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 16/35] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:41 2022] GET /identity/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:41.738183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b6c98-bf89-4e54-b19a-520c16fcbea3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:41.768127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 20/36] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:14:41 2022] POST /identity/v3/projects => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:14:41.768973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d71e0-58f6-4620-afa2-d96bae762153 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:41.861645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:41.882659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b6c98-bf89-4e54-b19a-520c16fcbea3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:41.908135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d71e0-58f6-4620-afa2-d96bae762153 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:41.918788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:41.923032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df08ef-de2d-4acf-a8fe-74e13b1e3dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:41.924338 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df08ef-de2d-4acf-a8fe-74e13b1e3dec None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.925173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df08ef-de2d-4acf-a8fe-74e13b1e3dec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.925599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df08ef-de2d-4acf-a8fe-74e13b1e3dec None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.926522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df08ef-de2d-4acf-a8fe-74e13b1e3dec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:41.928989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df08ef-de2d-4acf-a8fe-74e13b1e3dec None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:41.953085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bf0f2-a521-46dc-9902-439a6a0762fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:41.953645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf0f2-a521-46dc-9902-439a6a0762fd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.953923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf0f2-a521-46dc-9902-439a6a0762fd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.954197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bf0f2-a521-46dc-9902-439a6a0762fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.955455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf0f2-a521-46dc-9902-439a6a0762fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:41.957040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bf0f2-a521-46dc-9902-439a6a0762fd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:41.958658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 21/37] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:41.960774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3f3f0-2c02-4024-a947-26e52500a59a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:41.961062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3f3f0-2c02-4024-a947-26e52500a59a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:41.961334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3f3f0-2c02-4024-a947-26e52500a59a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:41.962917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 22/38] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:41.964820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3f3f0-2c02-4024-a947-26e52500a59a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:42.062530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 17/39] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Aug 11 16:14:41 2022] POST /identity/v3/roles => generated 225 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:14:42.064498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff4e1a-fea8-4ddc-92c4-fca78262728d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:42.064928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff4e1a-fea8-4ddc-92c4-fca78262728d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:42.065408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff4e1a-fea8-4ddc-92c4-fca78262728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:42.100537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2bff4e1a-fea8-4ddc-92c4-fca78262728d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:42.102316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2bff4e1a-fea8-4ddc-92c4-fca78262728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:42.112375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3f3f0-2c02-4024-a947-26e52500a59a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:42.155527 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f50ace-2a9a-4087-9388-a3e379d2d710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:42.156727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f50ace-2a9a-4087-9388-a3e379d2d710 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:42.157215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f50ace-2a9a-4087-9388-a3e379d2d710 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:42.157672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f50ace-2a9a-4087-9388-a3e379d2d710 None admin] PATH_INFO: `/v3/domains/default/users/ddf38f7f3d7248769f04c04a1b75ce6c/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:42.160210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f50ace-2a9a-4087-9388-a3e379d2d710 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ddf38f7f3d7248769f04c04a1b75ce6c, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:42.162740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f50ace-2a9a-4087-9388-a3e379d2d710 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:42.183541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 23/40] 10.222.0.43 () {58 vars in 1361 bytes} [Thu Aug 11 16:14:41 2022] PUT /identity/v3/domains/default/users/ddf38f7f3d7248769f04c04a1b75ce6c/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:14:42.194523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 18/41] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:42.222003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f50ace-2a9a-4087-9388-a3e379d2d710 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:42.274534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:42.332500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170eb2f-6f26-4fbc-b246-4f51caa42e6e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:42.332788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170eb2f-6f26-4fbc-b246-4f51caa42e6e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:42.333071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170eb2f-6f26-4fbc-b246-4f51caa42e6e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:42.335434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 19/42] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:14:42.341217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b652b0-07dd-4ed9-a719-e346a3a5e7f7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:42.341862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b652b0-07dd-4ed9-a719-e346a3a5e7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:42.342342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b652b0-07dd-4ed9-a719-e346a3a5e7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:42.363067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f50ace-2a9a-4087-9388-a3e379d2d710 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:42.363251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c3b652b0-07dd-4ed9-a719-e346a3a5e7f7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:42.364492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c3b652b0-07dd-4ed9-a719-e346a3a5e7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:42.402849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b16b54-8432-48b9-bdb9-d5f8c6230620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:42.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b16b54-8432-48b9-bdb9-d5f8c6230620 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:42.403734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b16b54-8432-48b9-bdb9-d5f8c6230620 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:42.404007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b16b54-8432-48b9-bdb9-d5f8c6230620 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:42.404725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b16b54-8432-48b9-bdb9-d5f8c6230620 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:42.406105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b16b54-8432-48b9-bdb9-d5f8c6230620 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:42.421017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 24/43] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Aug 11 16:14:42 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:14:42.442924 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 20/44] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:14:42.467345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b16b54-8432-48b9-bdb9-d5f8c6230620 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:42.575212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:42.584522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b16b54-8432-48b9-bdb9-d5f8c6230620 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:42.620148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12bb51-f8e7-4b8f-b0a9-bb7c3be4f647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:42.620777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12bb51-f8e7-4b8f-b0a9-bb7c3be4f647 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:42.621054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12bb51-f8e7-4b8f-b0a9-bb7c3be4f647 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:42.621320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12bb51-f8e7-4b8f-b0a9-bb7c3be4f647 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:42.622458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12bb51-f8e7-4b8f-b0a9-bb7c3be4f647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:42.623966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12bb51-f8e7-4b8f-b0a9-bb7c3be4f647 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:42.625632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 25/45] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:42.632157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b652b0-07dd-4ed9-a719-e346a3a5e7f7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:42.640896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:42.733109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b652b0-07dd-4ed9-a719-e346a3a5e7f7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:42.766341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde288d-c8b0-4d61-adf7-9a835b23aa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:42.766960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde288d-c8b0-4d61-adf7-9a835b23aa5c None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:42.767245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde288d-c8b0-4d61-adf7-9a835b23aa5c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:42.767520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde288d-c8b0-4d61-adf7-9a835b23aa5c None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:42.768214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde288d-c8b0-4d61-adf7-9a835b23aa5c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:42.769789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde288d-c8b0-4d61-adf7-9a835b23aa5c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:42.798211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 21/46] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:14:42 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:14:44.236264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40834-7e95-40ee-b597-2d4f7c104a19 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:44.236804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40834-7e95-40ee-b597-2d4f7c104a19 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:44.236804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40834-7e95-40ee-b597-2d4f7c104a19 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:44.238154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 26/47] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:44.244314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1de34-22de-4f38-bfd7-3a775362b56d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:44.244702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1de34-22de-4f38-bfd7-3a775362b56d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:44.244949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1de34-22de-4f38-bfd7-3a775362b56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:44.263415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-71a1de34-22de-4f38-bfd7-3a775362b56d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:44.264626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-71a1de34-22de-4f38-bfd7-3a775362b56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:44.317464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:44.332371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 22/48] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:44.355482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c40834-7e95-40ee-b597-2d4f7c104a19 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:44.456924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c40834-7e95-40ee-b597-2d4f7c104a19 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:44.490050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca0a37-3505-4485-ace3-8f2dc7b698f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:44.491439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca0a37-3505-4485-ace3-8f2dc7b698f3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:44.491731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca0a37-3505-4485-ace3-8f2dc7b698f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:44.492001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca0a37-3505-4485-ace3-8f2dc7b698f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:44.502215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca0a37-3505-4485-ace3-8f2dc7b698f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-b8ca0a37-3505-4485-ace3-8f2dc7b698f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:44.507553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:44.508518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:14:44.509353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 27/49] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:44.516232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1de34-22de-4f38-bfd7-3a775362b56d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:44.614332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1de34-22de-4f38-bfd7-3a775362b56d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:44.648654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcd1b2-532d-4282-8946-89e5ef5e8181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:44.649271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd1b2-532d-4282-8946-89e5ef5e8181 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:44.649556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd1b2-532d-4282-8946-89e5ef5e8181 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:44.649832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd1b2-532d-4282-8946-89e5ef5e8181 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:44.650495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcd1b2-532d-4282-8946-89e5ef5e8181 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:44.650799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcd1b2-532d-4282-8946-89e5ef5e8181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:44.652221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbcd1b2-532d-4282-8946-89e5ef5e8181 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:44.664623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 23/50] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:44.670329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca0a37-3505-4485-ace3-8f2dc7b698f3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:44.695562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:44.768754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca0a37-3505-4485-ace3-8f2dc7b698f3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:44.802229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3d75-c96f-4f02-bd7a-4ac270651747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:44.802892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3d75-c96f-4f02-bd7a-4ac270651747 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:44.803172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3d75-c96f-4f02-bd7a-4ac270651747 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:44.803262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357b90b-8491-4cf0-8fd1-6e6c1011f543 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:44.803511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3d75-c96f-4f02-bd7a-4ac270651747 None admin] PATH_INFO: `/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:44.803596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357b90b-8491-4cf0-8fd1-6e6c1011f543 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:44.803813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357b90b-8491-4cf0-8fd1-6e6c1011f543 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:44.804616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3d75-c96f-4f02-bd7a-4ac270651747 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf38f7f3d7248769f04c04a1b75ce6c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:44.805186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 24/51] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:14:44.806070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3d75-c96f-4f02-bd7a-4ac270651747 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:44.807644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 28/52] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:44 2022] GET /identity/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:44.812863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3d75-c96f-4f02-bd7a-4ac270651747 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:44.813071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ab496-dad6-4f8f-9f39-02e01b6a9dae None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:44.813354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ab496-dad6-4f8f-9f39-02e01b6a9dae None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:44.813893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ab496-dad6-4f8f-9f39-02e01b6a9dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:44.831209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4e7ab496-dad6-4f8f-9f39-02e01b6a9dae None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:44.832434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4e7ab496-dad6-4f8f-9f39-02e01b6a9dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:44.877439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:44.898311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 25/53] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:44.907918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:44.917299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3d75-c96f-4f02-bd7a-4ac270651747 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:44.920768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ab496-dad6-4f8f-9f39-02e01b6a9dae None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:44.952312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279016ae-9bca-4a7a-a0b0-3dc219c16d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:44.952932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279016ae-9bca-4a7a-a0b0-3dc219c16d96 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:44.953208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279016ae-9bca-4a7a-a0b0-3dc219c16d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:44.953477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279016ae-9bca-4a7a-a0b0-3dc219c16d96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:44.954587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279016ae-9bca-4a7a-a0b0-3dc219c16d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:44.956026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279016ae-9bca-4a7a-a0b0-3dc219c16d96 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:44.957604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 29/54] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:44.963506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279016ae-9bca-4a7a-a0b0-3dc219c16d96 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:45.019781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ab496-dad6-4f8f-9f39-02e01b6a9dae None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:45.054916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490dc36-eefc-4364-b5c9-fa8c48942c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:45.055488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490dc36-eefc-4364-b5c9-fa8c48942c72 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:45.055772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490dc36-eefc-4364-b5c9-fa8c48942c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:45.056429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490dc36-eefc-4364-b5c9-fa8c48942c72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:45.057277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490dc36-eefc-4364-b5c9-fa8c48942c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:45.058725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490dc36-eefc-4364-b5c9-fa8c48942c72 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:45.060352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 26/55] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:45.065815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279016ae-9bca-4a7a-a0b0-3dc219c16d96 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:45.068107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490dc36-eefc-4364-b5c9-fa8c48942c72 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:45.101592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009919c-b26a-4236-a139-3515e98f79aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:45.102198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009919c-b26a-4236-a139-3515e98f79aa None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:45.102482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009919c-b26a-4236-a139-3515e98f79aa None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:45.102754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009919c-b26a-4236-a139-3515e98f79aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:45.103477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009919c-b26a-4236-a139-3515e98f79aa None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.domain.id=default, user.id=ddf38f7f3d7248769f04c04a1b75ce6c) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:45.103784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009919c-b26a-4236-a139-3515e98f79aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:45.105252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009919c-b26a-4236-a139-3515e98f79aa None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:45.111322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 30/56] 10.222.0.43 () {56 vars in 1322 bytes} [Thu Aug 11 16:14:44 2022] GET /identity/v3/role_assignments?user.id=ddf38f7f3d7248769f04c04a1b75ce6c&scope.domain.id=default&role.id=29fd031edd2b4f65aed8942853150f92 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:45.118237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:45.173173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490dc36-eefc-4364-b5c9-fa8c48942c72 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:45.208546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fd5ee-a8ca-4d29-9672-eb42dcb52e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:45.209187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fd5ee-a8ca-4d29-9672-eb42dcb52e2d None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:45.209466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fd5ee-a8ca-4d29-9672-eb42dcb52e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:45.209738 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fd5ee-a8ca-4d29-9672-eb42dcb52e2d None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:45.210423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fd5ee-a8ca-4d29-9672-eb42dcb52e2d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:45.211890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fd5ee-a8ca-4d29-9672-eb42dcb52e2d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:45.241213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 27/57] 10.222.0.43 () {60 vars in 1140 bytes} [Thu Aug 11 16:14:45 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:14:47.224793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24635a69-c40f-4865-aa4c-547d2b792b16 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:47.225297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24635a69-c40f-4865-aa4c-547d2b792b16 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:47.225381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24635a69-c40f-4865-aa4c-547d2b792b16 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:47.226701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 31/58] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:47.231974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752981d-5d57-4232-af28-46e58a209bb6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:47.232417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752981d-5d57-4232-af28-46e58a209bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:47.232585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752981d-5d57-4232-af28-46e58a209bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:47.250756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2752981d-5d57-4232-af28-46e58a209bb6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:47.251934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2752981d-5d57-4232-af28-46e58a209bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:47.316271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 28/59] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:47.340072 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24635a69-c40f-4865-aa4c-547d2b792b16 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:47.366297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:47.439688 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24635a69-c40f-4865-aa4c-547d2b792b16 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:47.470826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15ead9-1a71-4a31-828e-883a06ae4baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:47.471441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15ead9-1a71-4a31-828e-883a06ae4baf None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:47.471718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15ead9-1a71-4a31-828e-883a06ae4baf None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:47.471988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15ead9-1a71-4a31-828e-883a06ae4baf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:47.473112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15ead9-1a71-4a31-828e-883a06ae4baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:47.475259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15ead9-1a71-4a31-828e-883a06ae4baf None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:47.476834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 32/60] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:47.482414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752981d-5d57-4232-af28-46e58a209bb6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:47.511525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:47.579429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752981d-5d57-4232-af28-46e58a209bb6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:47.614167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ccb638-64a1-4e0f-b08e-016a69389ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:47.614824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccb638-64a1-4e0f-b08e-016a69389ed3 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:47.615157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccb638-64a1-4e0f-b08e-016a69389ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:47.615972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ccb638-64a1-4e0f-b08e-016a69389ed3 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:47.616895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccb638-64a1-4e0f-b08e-016a69389ed3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:47.618417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ccb638-64a1-4e0f-b08e-016a69389ed3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:47.643677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 29/61] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:14:47 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:14:50.647238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb225f1d-f565-48fa-a271-8c05a293e794 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:50.647743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb225f1d-f565-48fa-a271-8c05a293e794 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:50.647815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb225f1d-f565-48fa-a271-8c05a293e794 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:50.649134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 33/62] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:50.654398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c63e-a3d4-4159-a540-bdcba24ae871 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:50.655137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c63e-a3d4-4159-a540-bdcba24ae871 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:50.655550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76c63e-a3d4-4159-a540-bdcba24ae871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:50.677067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3f76c63e-a3d4-4159-a540-bdcba24ae871 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:50.679333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3f76c63e-a3d4-4159-a540-bdcba24ae871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:50.786367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 30/63] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:14:50.828230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb225f1d-f565-48fa-a271-8c05a293e794 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:50.955631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:50.966109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb225f1d-f565-48fa-a271-8c05a293e794 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:51.017890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9974ac-daeb-4e8f-ab85-fa6b0fdf46ae None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.018490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9974ac-daeb-4e8f-ab85-fa6b0fdf46ae None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.018913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae615fbc-3bc9-48f2-a326-b7af8aabd569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:51.019133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9974ac-daeb-4e8f-ab85-fa6b0fdf46ae None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.019552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615fbc-3bc9-48f2-a326-b7af8aabd569 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.019831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615fbc-3bc9-48f2-a326-b7af8aabd569 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.020134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae615fbc-3bc9-48f2-a326-b7af8aabd569 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.021873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 31/64] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:51.023122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdcb73-fb3f-48b6-a13c-aa71dc1cd0b7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.023609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdcb73-fb3f-48b6-a13c-aa71dc1cd0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.024077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdcb73-fb3f-48b6-a13c-aa71dc1cd0b7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.025524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 32/65] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:51.030287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cad9a-ddf1-4035-bd9e-2020d30da7e7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.030574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cad9a-ddf1-4035-bd9e-2020d30da7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.030848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711cad9a-ddf1-4035-bd9e-2020d30da7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.037575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae615fbc-3bc9-48f2-a326-b7af8aabd569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-ae615fbc-3bc9-48f2-a326-b7af8aabd569 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:51.044373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:51.045239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:14:51.046010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 34/66] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:51.046718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b124-03ea-4269-aeae-ba090a49e228 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.047533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b124-03ea-4269-aeae-ba090a49e228 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.047533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b124-03ea-4269-aeae-ba090a49e228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.061495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-711cad9a-ddf1-4035-bd9e-2020d30da7e7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:51.062974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-711cad9a-ddf1-4035-bd9e-2020d30da7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:51.078844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5b77b124-03ea-4269-aeae-ba090a49e228 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:51.080473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:51.080769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5b77b124-03ea-4269-aeae-ba090a49e228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:51.148125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 33/67] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:51.148125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cad9a-ddf1-4035-bd9e-2020d30da7e7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:51.187440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 35/68] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:51.189120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b124-03ea-4269-aeae-ba090a49e228 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:51.277415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711cad9a-ddf1-4035-bd9e-2020d30da7e7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:51.292740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:51.320922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b124-03ea-4269-aeae-ba090a49e228 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:51.330073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:51.331317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.331800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.332579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.334462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:51.335002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:51.337174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:51.347490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 34/69] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:51.348611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:51.357762 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3399c-465a-4df9-b5f6-38357f42fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:51.358394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3399c-465a-4df9-b5f6-38357f42fa79 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.358671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3399c-465a-4df9-b5f6-38357f42fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.358940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3399c-465a-4df9-b5f6-38357f42fa79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.361910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:51.370681 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3399c-465a-4df9-b5f6-38357f42fa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-7be3399c-465a-4df9-b5f6-38357f42fa79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:51.375835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:51.376882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 36/70] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:51.383357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b30ed-a3fe-4c4a-a5cc-854b215154cb None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.384155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b30ed-a3fe-4c4a-a5cc-854b215154cb None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.384155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b30ed-a3fe-4c4a-a5cc-854b215154cb None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.384155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 37/71] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:51.384155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b30ed-a3fe-4c4a-a5cc-854b215154cb None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:51.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a19a34-f31d-4314-b573-5b8ad5e51f78 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:51.497828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b30ed-a3fe-4c4a-a5cc-854b215154cb None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:51.502184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c07b3-6e87-4142-9019-07e2d0275edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:51.502819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c07b3-6e87-4142-9019-07e2d0275edc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.503099 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c07b3-6e87-4142-9019-07e2d0275edc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.503368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c07b3-6e87-4142-9019-07e2d0275edc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.514106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c07b3-6e87-4142-9019-07e2d0275edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-dc5c07b3-6e87-4142-9019-07e2d0275edc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:51.519261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:51.520072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 35/72] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:51.526264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c07b3-6e87-4142-9019-07e2d0275edc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:51.533002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01612a-d43e-48ed-99d2-8b1eb8e2f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:51.533571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01612a-d43e-48ed-99d2-8b1eb8e2f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.533852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01612a-d43e-48ed-99d2-8b1eb8e2f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.534128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01612a-d43e-48ed-99d2-8b1eb8e2f0f9 None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.546563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:51.549268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01612a-d43e-48ed-99d2-8b1eb8e2f0f9 None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:51.551085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01612a-d43e-48ed-99d2-8b1eb8e2f0f9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:51.553573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 38/73] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:51.554800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658eddb-de1d-4465-b674-115c820f685b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.555095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658eddb-de1d-4465-b674-115c820f685b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.555372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658eddb-de1d-4465-b674-115c820f685b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.572574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d658eddb-de1d-4465-b674-115c820f685b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:51.573787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d658eddb-de1d-4465-b674-115c820f685b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:51.591649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:51.625119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c07b3-6e87-4142-9019-07e2d0275edc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:51.640714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 39/74] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:51.641936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d278c-e406-45ef-9435-4d737c5d741a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.642226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d278c-e406-45ef-9435-4d737c5d741a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.642499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d278c-e406-45ef-9435-4d737c5d741a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.643736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 40/75] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:51.649025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d278c-e406-45ef-9435-4d737c5d741a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:51.666994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:51.667610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.667892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.668195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.668859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:51.669163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:51.670517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:51.679986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 36/76] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:51.680836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:51.762168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d278c-e406-45ef-9435-4d737c5d741a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:51.781772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93b8a6-250c-481d-8c53-c40bde2a5d13 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:51.791299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:51.797693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f61824-f698-4da3-8e74-4a7587a6f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:51.798279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f61824-f698-4da3-8e74-4a7587a6f84e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.798562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f61824-f698-4da3-8e74-4a7587a6f84e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.798825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f61824-f698-4da3-8e74-4a7587a6f84e None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.799484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f61824-f698-4da3-8e74-4a7587a6f84e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:51.799783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f61824-f698-4da3-8e74-4a7587a6f84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:51.801212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f61824-f698-4da3-8e74-4a7587a6f84e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:51.809656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 41/77] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:51.810904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:51.815664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb03cd9-7bbe-4dd0-8cc1-bea454cb3f5f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.815998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb03cd9-7bbe-4dd0-8cc1-bea454cb3f5f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.816249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb03cd9-7bbe-4dd0-8cc1-bea454cb3f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.817839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab4212-15b2-42f1-9f7b-fd4cd0ba1265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:51.818641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab4212-15b2-42f1-9f7b-fd4cd0ba1265 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:51.818914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab4212-15b2-42f1-9f7b-fd4cd0ba1265 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:51.819188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab4212-15b2-42f1-9f7b-fd4cd0ba1265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:51.833219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-bfb03cd9-7bbe-4dd0-8cc1-bea454cb3f5f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:51.834717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-bfb03cd9-7bbe-4dd0-8cc1-bea454cb3f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:51.885260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab4212-15b2-42f1-9f7b-fd4cd0ba1265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:51.886728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab4212-15b2-42f1-9f7b-fd4cd0ba1265 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:51.901784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 42/78] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:51.912286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb03cd9-7bbe-4dd0-8cc1-bea454cb3f5f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:51.982259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 37/79] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:14:51.982259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab4212-15b2-42f1-9f7b-fd4cd0ba1265 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.011920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.015351 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb03cd9-7bbe-4dd0-8cc1-bea454cb3f5f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.038247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.050501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300f532-afdd-40eb-8918-87ce7648315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.051156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f532-afdd-40eb-8918-87ce7648315a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.051538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f532-afdd-40eb-8918-87ce7648315a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.051707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300f532-afdd-40eb-8918-87ce7648315a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.063140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300f532-afdd-40eb-8918-87ce7648315a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-5300f532-afdd-40eb-8918-87ce7648315a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:52.068272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:52.069235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:14:52.070035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 43/80] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:52.070687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300f532-afdd-40eb-8918-87ce7648315a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.086772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab4212-15b2-42f1-9f7b-fd4cd0ba1265 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.121455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2397d2-048a-41d0-9824-5c34b82e06c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.122079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2397d2-048a-41d0-9824-5c34b82e06c8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.122357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2397d2-048a-41d0-9824-5c34b82e06c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.122628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2397d2-048a-41d0-9824-5c34b82e06c8 None admin] PATH_INFO: `/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.123749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2397d2-048a-41d0-9824-5c34b82e06c8 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf38f7f3d7248769f04c04a1b75ce6c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.125220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2397d2-048a-41d0-9824-5c34b82e06c8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.126805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 38/81] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:51 2022] GET /identity/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:52.127564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2397d2-048a-41d0-9824-5c34b82e06c8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.170915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300f532-afdd-40eb-8918-87ce7648315a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.204214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa607f-e638-424e-a58f-aab21e8abe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.204838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa607f-e638-424e-a58f-aab21e8abe50 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.205116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa607f-e638-424e-a58f-aab21e8abe50 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.205389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa607f-e638-424e-a58f-aab21e8abe50 None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.206523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa607f-e638-424e-a58f-aab21e8abe50 None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.207960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa607f-e638-424e-a58f-aab21e8abe50 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.210104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 44/82] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:52.210104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa607f-e638-424e-a58f-aab21e8abe50 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.225247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.229445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2397d2-048a-41d0-9824-5c34b82e06c8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.259515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.262895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64141ab3-91bb-4d25-9d57-f7dbf853d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.263525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141ab3-91bb-4d25-9d57-f7dbf853d7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.263840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141ab3-91bb-4d25-9d57-f7dbf853d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.264071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64141ab3-91bb-4d25-9d57-f7dbf853d7a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.265261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64141ab3-91bb-4d25-9d57-f7dbf853d7a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.266703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64141ab3-91bb-4d25-9d57-f7dbf853d7a6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.268294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 39/83] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:52.269060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64141ab3-91bb-4d25-9d57-f7dbf853d7a6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.309825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa607f-e638-424e-a58f-aab21e8abe50 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.342890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfa8c1-9b29-472c-a1da-f5db8ddcc301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.343518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfa8c1-9b29-472c-a1da-f5db8ddcc301 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.343793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfa8c1-9b29-472c-a1da-f5db8ddcc301 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.344065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfa8c1-9b29-472c-a1da-f5db8ddcc301 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.345215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfa8c1-9b29-472c-a1da-f5db8ddcc301 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.346660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfa8c1-9b29-472c-a1da-f5db8ddcc301 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.348318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 45/84] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:52.349070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfa8c1-9b29-472c-a1da-f5db8ddcc301 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.373802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64141ab3-91bb-4d25-9d57-f7dbf853d7a6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.407859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.408497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.408834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.409114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.409776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:52.410079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.411446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.419896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 40/85] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:52.420713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.447786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfa8c1-9b29-472c-a1da-f5db8ddcc301 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.451460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.465202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.482553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc434c-5fcf-4c37-82dc-7c69cbcf150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.483437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc434c-5fcf-4c37-82dc-7c69cbcf150c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.483724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc434c-5fcf-4c37-82dc-7c69cbcf150c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.483997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc434c-5fcf-4c37-82dc-7c69cbcf150c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.521254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f4357-f9f7-45cf-91d2-c8fb0c682225 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.550199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc434c-5fcf-4c37-82dc-7c69cbcf150c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.551661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc434c-5fcf-4c37-82dc-7c69cbcf150c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.554397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438a46d-156b-4aef-baa6-84546d1d52b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.555001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a46d-156b-4aef-baa6-84546d1d52b5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.555279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a46d-156b-4aef-baa6-84546d1d52b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.555552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438a46d-156b-4aef-baa6-84546d1d52b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.623139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438a46d-156b-4aef-baa6-84546d1d52b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.624548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438a46d-156b-4aef-baa6-84546d1d52b5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.647977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 46/86] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:14:52.648834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc434c-5fcf-4c37-82dc-7c69cbcf150c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.650649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.667194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.729063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 41/87] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:14:52.729063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438a46d-156b-4aef-baa6-84546d1d52b5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.752536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc434c-5fcf-4c37-82dc-7c69cbcf150c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.784939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a045c-2d6c-4ea5-bdaf-6725c005f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.785534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a045c-2d6c-4ea5-bdaf-6725c005f6f6 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.786097 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a045c-2d6c-4ea5-bdaf-6725c005f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.786377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a045c-2d6c-4ea5-bdaf-6725c005f6f6 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.787670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a045c-2d6c-4ea5-bdaf-6725c005f6f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.789111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a045c-2d6c-4ea5-bdaf-6725c005f6f6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.816061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 47/88] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:14:52 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:14:52.816958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a045c-2d6c-4ea5-bdaf-6725c005f6f6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.824500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438a46d-156b-4aef-baa6-84546d1d52b5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:52.854188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.863263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:52.863883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:52.864194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:52.864473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:52.865818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] RBAC: Adding query filter params (role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=d38685d4e3e64d9ba662890b9147cac3, user.id=0898fbca99fc478e9cab823c368a904f) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:52.866153 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:52.867640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:52.877165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 42/89] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/role_assignments?user.id=0898fbca99fc478e9cab823c368a904f&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:52.877971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:52.939441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:52.988345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a045c-2d6c-4ea5-bdaf-6725c005f6f6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:53.026802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314184b-43af-44a2-8762-7d3c829e156e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:53.043589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c9dfb-48e5-4756-831a-6f78170cde90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:53.043982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c9dfb-48e5-4756-831a-6f78170cde90 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:53.044331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c9dfb-48e5-4756-831a-6f78170cde90 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:53.044620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c9dfb-48e5-4756-831a-6f78170cde90 None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:53.045884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c9dfb-48e5-4756-831a-6f78170cde90 None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:53.047379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c9dfb-48e5-4756-831a-6f78170cde90 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:53.049297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 48/90] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:53.055014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c9dfb-48e5-4756-831a-6f78170cde90 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:53.087190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89907ae7-60ec-418e-b50c-96b2ecd23901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:53.087875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89907ae7-60ec-418e-b50c-96b2ecd23901 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:53.088211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89907ae7-60ec-418e-b50c-96b2ecd23901 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:53.088487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89907ae7-60ec-418e-b50c-96b2ecd23901 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:53.089627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89907ae7-60ec-418e-b50c-96b2ecd23901 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:53.091686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89907ae7-60ec-418e-b50c-96b2ecd23901 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:53.094042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 43/91] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:52 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:53.094042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89907ae7-60ec-418e-b50c-96b2ecd23901 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:53.170560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:53.197264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c9dfb-48e5-4756-831a-6f78170cde90 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:53.234083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89907ae7-60ec-418e-b50c-96b2ecd23901 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:53.245017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77340d8-2353-4b68-adaa-a532d4bbc242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:53.245658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77340d8-2353-4b68-adaa-a532d4bbc242 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:53.245938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77340d8-2353-4b68-adaa-a532d4bbc242 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:53.246213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77340d8-2353-4b68-adaa-a532d4bbc242 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:53.247794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77340d8-2353-4b68-adaa-a532d4bbc242 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:53.249647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77340d8-2353-4b68-adaa-a532d4bbc242 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:53.252703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 49/92] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:53 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:53.253723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77340d8-2353-4b68-adaa-a532d4bbc242 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:53.271261 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:53.277133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75da8f6-d5cf-4ef7-aab3-24ecb11449ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:53.278276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da8f6-d5cf-4ef7-aab3-24ecb11449ba None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:53.278738 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da8f6-d5cf-4ef7-aab3-24ecb11449ba None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:53.279367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da8f6-d5cf-4ef7-aab3-24ecb11449ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:53.357773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75da8f6-d5cf-4ef7-aab3-24ecb11449ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:53.359367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75da8f6-d5cf-4ef7-aab3-24ecb11449ba None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:53.376977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77340d8-2353-4b68-adaa-a532d4bbc242 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:53.416587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:53.417485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:53.417961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:53.418547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:53.419582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=d38685d4e3e64d9ba662890b9147cac3, user.id=ddf38f7f3d7248769f04c04a1b75ce6c) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:53.420086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:53.422065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:53.422470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:53.431454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 50/93] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:14:53 2022] GET /identity/v3/role_assignments?user.id=ddf38f7f3d7248769f04c04a1b75ce6c&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=29fd031edd2b4f65aed8942853150f92 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:53.432818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:53.468539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 44/94] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:14:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:14:53.474202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75da8f6-d5cf-4ef7-aab3-24ecb11449ba None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:53.523799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:53.554244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d95f29-874b-402d-9202-cbd7437a0f5c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:53.580785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75da8f6-d5cf-4ef7-aab3-24ecb11449ba None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:53.594952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:53.595969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:53.596533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:53.597001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:53.598085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] RBAC: Adding query filter params (role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=d38685d4e3e64d9ba662890b9147cac3, user.id=0898fbca99fc478e9cab823c368a904f) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:53.598640 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:53.600986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:53.608491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 51/95] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:14:53 2022] GET /identity/v3/role_assignments?user.id=0898fbca99fc478e9cab823c368a904f&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:53.621941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97980e-5500-4fd3-bb07-903f9aa78728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:53.622605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97980e-5500-4fd3-bb07-903f9aa78728 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:53.622890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97980e-5500-4fd3-bb07-903f9aa78728 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:53.623162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97980e-5500-4fd3-bb07-903f9aa78728 None admin] PATH_INFO: `/v3/projects/8f143e4d2e324cd688773436ad8d7a49` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:53.624454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97980e-5500-4fd3-bb07-903f9aa78728 None admin] RBAC: Authorizing `identity:get_project(project_id=8f143e4d2e324cd688773436ad8d7a49)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:53.625927 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97980e-5500-4fd3-bb07-903f9aa78728 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:53.627878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 45/96] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:53 2022] GET /identity/v3/projects/8f143e4d2e324cd688773436ad8d7a49 => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:14:53.633969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:53.765750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79b436-30b3-4601-92be-a8ad5c7864a4 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:53.809105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4302f4-188d-49c3-8db8-f1dbde683c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:53.810091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4302f4-188d-49c3-8db8-f1dbde683c31 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:53.810662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4302f4-188d-49c3-8db8-f1dbde683c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:53.811131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4302f4-188d-49c3-8db8-f1dbde683c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:53.812226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4302f4-188d-49c3-8db8-f1dbde683c31 None admin] RBAC: Adding query filter params (role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=8f143e4d2e324cd688773436ad8d7a49, user.id=0898fbca99fc478e9cab823c368a904f) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:53.812738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4302f4-188d-49c3-8db8-f1dbde683c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:53.814677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4302f4-188d-49c3-8db8-f1dbde683c31 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:53.822227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 52/97] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:14:53 2022] GET /identity/v3/role_assignments?user.id=0898fbca99fc478e9cab823c368a904f&scope.project.id=8f143e4d2e324cd688773436ad8d7a49&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:55.524232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204dcb6-9f75-4a78-834d-d6d7a61a67e2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:55.525065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204dcb6-9f75-4a78-834d-d6d7a61a67e2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:55.525494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204dcb6-9f75-4a78-834d-d6d7a61a67e2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:55.527251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 46/98] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:55.535996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0d0b-525a-4a6a-9fe0-cd87398e4135 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:55.537104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0d0b-525a-4a6a-9fe0-cd87398e4135 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:55.537104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c0d0b-525a-4a6a-9fe0-cd87398e4135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:55.546684 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:55.567844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-545c0d0b-525a-4a6a-9fe0-cd87398e4135 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:55.569153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-545c0d0b-525a-4a6a-9fe0-cd87398e4135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:55.649018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 53/99] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:55.677727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204dcb6-9f75-4a78-834d-d6d7a61a67e2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:55.725415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:55.812801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204dcb6-9f75-4a78-834d-d6d7a61a67e2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:55.857814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf58df-cfee-44d4-89da-79bed7ceed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:55.858456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf58df-cfee-44d4-89da-79bed7ceed31 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:55.858737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf58df-cfee-44d4-89da-79bed7ceed31 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:55.859013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf58df-cfee-44d4-89da-79bed7ceed31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:55.873078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf58df-cfee-44d4-89da-79bed7ceed31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f4cf58df-cfee-44d4-89da-79bed7ceed31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:55.878341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:55.879207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:14:55.879966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 47/100] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:55.885763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c0d0b-525a-4a6a-9fe0-cd87398e4135 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:56.050928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c0d0b-525a-4a6a-9fe0-cd87398e4135 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:56.092554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:56.096066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1aa266-b99c-4f73-a235-0c190e2e2802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:56.097144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aa266-b99c-4f73-a235-0c190e2e2802 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.097781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aa266-b99c-4f73-a235-0c190e2e2802 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.098459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1aa266-b99c-4f73-a235-0c190e2e2802 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.099611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1aa266-b99c-4f73-a235-0c190e2e2802 None admin] RBAC: Adding query filter params (name=member) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:56.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1aa266-b99c-4f73-a235-0c190e2e2802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:56.101320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1aa266-b99c-4f73-a235-0c190e2e2802 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:56.110330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 54/101] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:14:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:56.121236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf58df-cfee-44d4-89da-79bed7ceed31 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:56.254009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:56.270230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf58df-cfee-44d4-89da-79bed7ceed31 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:56.291987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e324-acbc-4386-9350-21b3dcc27516 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.292629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e324-acbc-4386-9350-21b3dcc27516 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.293128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e324-acbc-4386-9350-21b3dcc27516 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.299093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 55/102] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:14:56.306043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47ef33-bee2-4e38-b6dd-1f3e9f03da3a None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.306673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47ef33-bee2-4e38-b6dd-1f3e9f03da3a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.307161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47ef33-bee2-4e38-b6dd-1f3e9f03da3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.336146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19af35b-ca2c-4281-8748-32305a53a56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:56.336146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19af35b-ca2c-4281-8748-32305a53a56b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.336146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19af35b-ca2c-4281-8748-32305a53a56b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.336146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19af35b-ca2c-4281-8748-32305a53a56b None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.337768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5e47ef33-bee2-4e38-b6dd-1f3e9f03da3a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:56.339199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5e47ef33-bee2-4e38-b6dd-1f3e9f03da3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:56.340829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19af35b-ca2c-4281-8748-32305a53a56b None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:56.342407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19af35b-ca2c-4281-8748-32305a53a56b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:56.348189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 48/103] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:56 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:56.350710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19af35b-ca2c-4281-8748-32305a53a56b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:56.521007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 56/104] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:14:56.522399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca232087-0a6a-4ed3-ac17-70aee674733b None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.522687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca232087-0a6a-4ed3-ac17-70aee674733b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.522955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca232087-0a6a-4ed3-ac17-70aee674733b None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.524536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 57/105] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:56.530341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313171-9a61-43b9-9eb8-b4d50d43405c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.530645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313171-9a61-43b9-9eb8-b4d50d43405c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.530899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0313171-9a61-43b9-9eb8-b4d50d43405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.569740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19af35b-ca2c-4281-8748-32305a53a56b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:56.571793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f0313171-9a61-43b9-9eb8-b4d50d43405c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:56.573043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f0313171-9a61-43b9-9eb8-b4d50d43405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:56.640190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400fd7d-c5c4-451b-b7c3-08ab320659a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:56.640765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400fd7d-c5c4-451b-b7c3-08ab320659a3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.641047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400fd7d-c5c4-451b-b7c3-08ab320659a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.641321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400fd7d-c5c4-451b-b7c3-08ab320659a3 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.643690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400fd7d-c5c4-451b-b7c3-08ab320659a3 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:56.645438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400fd7d-c5c4-451b-b7c3-08ab320659a3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:56.648767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 49/106] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:56 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:56.649569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400fd7d-c5c4-451b-b7c3-08ab320659a3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:56.683774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:56.696513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 58/107] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:56.697798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485a1cc-bcbd-469f-967d-fbaa2c55baa6 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.698087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485a1cc-bcbd-469f-967d-fbaa2c55baa6 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.698366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485a1cc-bcbd-469f-967d-fbaa2c55baa6 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.699772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 59/108] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:56.704407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485a1cc-bcbd-469f-967d-fbaa2c55baa6 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:56.768509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:56.778298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400fd7d-c5c4-451b-b7c3-08ab320659a3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:56.822866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1485a1cc-bcbd-469f-967d-fbaa2c55baa6 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:56.823869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154326d-f9ba-49c6-955d-00aa98ae4d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:56.824625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154326d-f9ba-49c6-955d-00aa98ae4d6b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.824960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154326d-f9ba-49c6-955d-00aa98ae4d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.825236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154326d-f9ba-49c6-955d-00aa98ae4d6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.836433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154326d-f9ba-49c6-955d-00aa98ae4d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1154326d-f9ba-49c6-955d-00aa98ae4d6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:56.842387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:56.843486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 50/109] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:14:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:56.844916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d377b-04bd-4141-ac42-f716fdc92499 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.845753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d377b-04bd-4141-ac42-f716fdc92499 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.845753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d377b-04bd-4141-ac42-f716fdc92499 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.846436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 51/110] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:56.847447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e9c6a-e50b-408f-a4f3-c5e8d2e7d7af None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.847724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e9c6a-e50b-408f-a4f3-c5e8d2e7d7af None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.847992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e9c6a-e50b-408f-a4f3-c5e8d2e7d7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.870504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5f9e9c6a-e50b-408f-a4f3-c5e8d2e7d7af None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:56.872326 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5f9e9c6a-e50b-408f-a4f3-c5e8d2e7d7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:56.874660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56de5f-755a-4812-aecc-c97925f7b807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:56.875333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56de5f-755a-4812-aecc-c97925f7b807 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:56.875621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56de5f-755a-4812-aecc-c97925f7b807 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:56.875898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d56de5f-755a-4812-aecc-c97925f7b807 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3/users/0898fbca99fc478e9cab823c368a904f/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:56.877886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56de5f-755a-4812-aecc-c97925f7b807 None admin] RBAC: Authorizing `identity:create_grant(project_id=d38685d4e3e64d9ba662890b9147cac3, user_id=0898fbca99fc478e9cab823c368a904f, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:56.879441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d56de5f-755a-4812-aecc-c97925f7b807 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:56.897849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 60/111] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:14:56 2022] PUT /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3/users/0898fbca99fc478e9cab823c368a904f/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:14:56.909115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56de5f-755a-4812-aecc-c97925f7b807 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:56.958435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 52/112] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:56.959333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e9c6a-e50b-408f-a4f3-c5e8d2e7d7af None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:56.984682 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:57.060899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d56de5f-755a-4812-aecc-c97925f7b807 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.083343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:57.105486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e9c6a-e50b-408f-a4f3-c5e8d2e7d7af None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.113716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2d326-9de0-4395-9e21-a2c733329901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.115208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d326-9de0-4395-9e21-a2c733329901 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.115663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d326-9de0-4395-9e21-a2c733329901 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.116827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2d326-9de0-4395-9e21-a2c733329901 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.132683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2d326-9de0-4395-9e21-a2c733329901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-dff2d326-9de0-4395-9e21-a2c733329901 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:57.137691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:57.138580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:14:57.139383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 61/113] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:57.141716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33b922-e52f-4209-b0eb-0c52319d876f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.142001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33b922-e52f-4209-b0eb-0c52319d876f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.142488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33b922-e52f-4209-b0eb-0c52319d876f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.151385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cd5ce-11d2-4b39-bbce-0de1a81d6989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.151922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd5ce-11d2-4b39-bbce-0de1a81d6989 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.152231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd5ce-11d2-4b39-bbce-0de1a81d6989 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.152580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd5ce-11d2-4b39-bbce-0de1a81d6989 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.153342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cd5ce-11d2-4b39-bbce-0de1a81d6989 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:57.153619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cd5ce-11d2-4b39-bbce-0de1a81d6989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:57.154982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cd5ce-11d2-4b39-bbce-0de1a81d6989 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:57.165623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5d33b922-e52f-4209-b0eb-0c52319d876f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:57.171470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 53/114] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:14:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:57.171675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5d33b922-e52f-4209-b0eb-0c52319d876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:57.171846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccffb39-48cb-434d-b107-6cba0343f84a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.171846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccffb39-48cb-434d-b107-6cba0343f84a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.171846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccffb39-48cb-434d-b107-6cba0343f84a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.171846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 54/115] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:57.171846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccffb39-48cb-434d-b107-6cba0343f84a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.253968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 62/116] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:57.254787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33b922-e52f-4209-b0eb-0c52319d876f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.281597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:57.311953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccffb39-48cb-434d-b107-6cba0343f84a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.351762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157eb0f-4d07-4536-8794-6b6b4e3c2cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.352660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157eb0f-4d07-4536-8794-6b6b4e3c2cbc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.353106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157eb0f-4d07-4536-8794-6b6b4e3c2cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.353521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157eb0f-4d07-4536-8794-6b6b4e3c2cbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.364296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157eb0f-4d07-4536-8794-6b6b4e3c2cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9157eb0f-4d07-4536-8794-6b6b4e3c2cbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:57.369752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:57.370835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:14:57.371732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:14:57.372911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33b922-e52f-4209-b0eb-0c52319d876f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.373819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 55/117] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:57.378306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157eb0f-4d07-4536-8794-6b6b4e3c2cbc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.382595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:57.420929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacf17d-e976-4429-ab16-c553a416d851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.421456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacf17d-e976-4429-ab16-c553a416d851 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.422149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacf17d-e976-4429-ab16-c553a416d851 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.422424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacf17d-e976-4429-ab16-c553a416d851 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.423092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacf17d-e976-4429-ab16-c553a416d851 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:57.423394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacf17d-e976-4429-ab16-c553a416d851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:57.425481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacf17d-e976-4429-ab16-c553a416d851 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:57.438638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 63/118] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:14:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:57.438638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1760ef-7095-4f75-84f1-7ea188310696 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.438638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1760ef-7095-4f75-84f1-7ea188310696 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.438638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1760ef-7095-4f75-84f1-7ea188310696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.455203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-7f1760ef-7095-4f75-84f1-7ea188310696 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:57.456496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7f1760ef-7095-4f75-84f1-7ea188310696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:57.487462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157eb0f-4d07-4536-8794-6b6b4e3c2cbc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.531278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a2bb-3fdf-4f20-90a9-95995eaf3d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.531914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a2bb-3fdf-4f20-90a9-95995eaf3d42 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.532243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a2bb-3fdf-4f20-90a9-95995eaf3d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.532521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a2bb-3fdf-4f20-90a9-95995eaf3d42 None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.533669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36a2bb-3fdf-4f20-90a9-95995eaf3d42 None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:57.541827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 64/119] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:57.541827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1760ef-7095-4f75-84f1-7ea188310696 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.542078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36a2bb-3fdf-4f20-90a9-95995eaf3d42 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:57.542078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 56/120] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:57 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:57.542078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a2bb-3fdf-4f20-90a9-95995eaf3d42 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.546992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:57.646529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:57.651793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a2bb-3fdf-4f20-90a9-95995eaf3d42 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.653601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1760ef-7095-4f75-84f1-7ea188310696 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.687845 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.688650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.688981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.689198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.689911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:57.690230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:57.691341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4b9c5-0921-4ca5-aa3d-3388ed3bcf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.691713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:57.691977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4b9c5-0921-4ca5-aa3d-3388ed3bcf24 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.692298 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4b9c5-0921-4ca5-aa3d-3388ed3bcf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.692606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4b9c5-0921-4ca5-aa3d-3388ed3bcf24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.694283 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4b9c5-0921-4ca5-aa3d-3388ed3bcf24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:57.695729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4b9c5-0921-4ca5-aa3d-3388ed3bcf24 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:57.697649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 65/121] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:14:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:57.698311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4b9c5-0921-4ca5-aa3d-3388ed3bcf24 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.702235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 57/122] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:57.703149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.772220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:57.810009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62764494-2f53-44d9-a46b-8353fa13048f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.813216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4b9c5-0921-4ca5-aa3d-3388ed3bcf24 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.846754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4339ae-e6fa-4103-9350-50975e90b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.847382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4339ae-e6fa-4103-9350-50975e90b30c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.847669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4339ae-e6fa-4103-9350-50975e90b30c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.847946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4339ae-e6fa-4103-9350-50975e90b30c None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.848215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c54d0-e02c-482c-9191-0715e6f0ae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:57.849392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c54d0-e02c-482c-9191-0715e6f0ae3b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:57.849626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4339ae-e6fa-4103-9350-50975e90b30c None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:57.849768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c54d0-e02c-482c-9191-0715e6f0ae3b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:57.849972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c54d0-e02c-482c-9191-0715e6f0ae3b None admin] PATH_INFO: `/v3/users/7b69638f9914428284970a3438274cfa` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:57.851260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4339ae-e6fa-4103-9350-50975e90b30c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:57.857716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 58/123] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:57 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:57.857716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4339ae-e6fa-4103-9350-50975e90b30c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.868969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c54d0-e02c-482c-9191-0715e6f0ae3b None admin] RBAC: Authorizing `identity:get_user(user_id=7b69638f9914428284970a3438274cfa)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:57.870431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c54d0-e02c-482c-9191-0715e6f0ae3b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:57.872350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 66/124] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:57 2022] GET /identity/v3/users/7b69638f9914428284970a3438274cfa => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:57.872945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c54d0-e02c-482c-9191-0715e6f0ae3b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:57.898741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:57.972785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4339ae-e6fa-4103-9350-50975e90b30c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:57.979994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c54d0-e02c-482c-9191-0715e6f0ae3b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:58.007735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a0e11-8727-4d85-bbc4-7d5471fb2abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.008394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0e11-8727-4d85-bbc4-7d5471fb2abc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.008674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0e11-8727-4d85-bbc4-7d5471fb2abc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.008947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0e11-8727-4d85-bbc4-7d5471fb2abc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.010839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:58.019275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568fad-2900-40b3-ae6f-eeaa18e5f610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.019886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568fad-2900-40b3-ae6f-eeaa18e5f610 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.020196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568fad-2900-40b3-ae6f-eeaa18e5f610 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.020477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a568fad-2900-40b3-ae6f-eeaa18e5f610 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.021706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a568fad-2900-40b3-ae6f-eeaa18e5f610 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:58.022365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a0e11-8727-4d85-bbc4-7d5471fb2abc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:58.023140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a568fad-2900-40b3-ae6f-eeaa18e5f610 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-448a0e11-8727-4d85-bbc4-7d5471fb2abc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:58.027562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:58.028458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:14:58.029264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 59/125] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:58.029903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a0e11-8727-4d85-bbc4-7d5471fb2abc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:58.062347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 67/126] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:14:57 2022] POST /identity/v3/projects => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:14:58.063168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568fad-2900-40b3-ae6f-eeaa18e5f610 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:58.165704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:58.194234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a0e11-8727-4d85-bbc4-7d5471fb2abc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:58.202213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a568fad-2900-40b3-ae6f-eeaa18e5f610 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:58.236500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe9853-9171-4535-a748-27200224b3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.237279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe9853-9171-4535-a748-27200224b3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.237402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe9853-9171-4535-a748-27200224b3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.237596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe9853-9171-4535-a748-27200224b3c2 None admin] PATH_INFO: `/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.239254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe9853-9171-4535-a748-27200224b3c2 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf38f7f3d7248769f04c04a1b75ce6c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:58.241911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe9853-9171-4535-a748-27200224b3c2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:58.244081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 60/127] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:58 2022] GET /identity/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:58.245188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe9853-9171-4535-a748-27200224b3c2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:58.247329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6a659-281c-440a-9629-bd25ac3aa3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.248425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6a659-281c-440a-9629-bd25ac3aa3bb None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.249751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6a659-281c-440a-9629-bd25ac3aa3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.250238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6a659-281c-440a-9629-bd25ac3aa3bb None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3/users/0898fbca99fc478e9cab823c368a904f/roles/3b5810bc1bb047f7851cef3afb74dcd0` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.252470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6a659-281c-440a-9629-bd25ac3aa3bb None admin] RBAC: Authorizing `identity:create_grant(project_id=d38685d4e3e64d9ba662890b9147cac3, user_id=0898fbca99fc478e9cab823c368a904f, role_id=3b5810bc1bb047f7851cef3afb74dcd0)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:58.253913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c6a659-281c-440a-9629-bd25ac3aa3bb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:58.269552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 68/128] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:14:58 2022] PUT /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3/users/0898fbca99fc478e9cab823c368a904f/roles/3b5810bc1bb047f7851cef3afb74dcd0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:14:58.270226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6a659-281c-440a-9629-bd25ac3aa3bb None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:58.328822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:58.379873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe9853-9171-4535-a748-27200224b3c2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:58.397960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6a659-281c-440a-9629-bd25ac3aa3bb None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:58.419305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dae2b-fa23-4012-8e9c-76babf41bb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.420035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dae2b-fa23-4012-8e9c-76babf41bb92 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.420358 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dae2b-fa23-4012-8e9c-76babf41bb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.420634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dae2b-fa23-4012-8e9c-76babf41bb92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.448630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.449266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.449553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.449834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.450528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:58.450834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:58.452288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:58.453944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:58.471811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 69/129] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:58.473409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:58.545758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dae2b-fa23-4012-8e9c-76babf41bb92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:58.547849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dae2b-fa23-4012-8e9c-76babf41bb92 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:58.606272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d52e3-77fa-465c-bd34-62355b3aace3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:58.634764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:58.677864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0fa63-51ce-4b3c-8392-4771ed983b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.679510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0fa63-51ce-4b3c-8392-4771ed983b61 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.679793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0fa63-51ce-4b3c-8392-4771ed983b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.682650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0fa63-51ce-4b3c-8392-4771ed983b61 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.683856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0fa63-51ce-4b3c-8392-4771ed983b61 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:58.685336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0fa63-51ce-4b3c-8392-4771ed983b61 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:58.687668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 70/130] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:58 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:58.688578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0fa63-51ce-4b3c-8392-4771ed983b61 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:58.715729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 61/131] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:14:58.717521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dae2b-fa23-4012-8e9c-76babf41bb92 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:58.815961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:58.885136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0fa63-51ce-4b3c-8392-4771ed983b61 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:58.897665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dae2b-fa23-4012-8e9c-76babf41bb92 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:58.950038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca25573-d22f-4fcf-b1f2-a454f3c312e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.951564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca25573-d22f-4fcf-b1f2-a454f3c312e7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.952252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca25573-d22f-4fcf-b1f2-a454f3c312e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.954191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca25573-d22f-4fcf-b1f2-a454f3c312e7 None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.956058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca25573-d22f-4fcf-b1f2-a454f3c312e7 None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:58.959042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca25573-d22f-4fcf-b1f2-a454f3c312e7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:58.960455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61ca86-136b-465c-9862-7d1b798984ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:58.961385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61ca86-136b-465c-9862-7d1b798984ab None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:58.961864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61ca86-136b-465c-9862-7d1b798984ab None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:58.963111 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61ca86-136b-465c-9862-7d1b798984ab None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3/users/ddf38f7f3d7248769f04c04a1b75ce6c/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:58.965073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 71/132] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:14:58 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:58.966474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61ca86-136b-465c-9862-7d1b798984ab None admin] RBAC: Authorizing `identity:create_grant(project_id=d38685d4e3e64d9ba662890b9147cac3, user_id=ddf38f7f3d7248769f04c04a1b75ce6c, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:58.967469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca25573-d22f-4fcf-b1f2-a454f3c312e7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:58.969130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61ca86-136b-465c-9862-7d1b798984ab None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:58.996895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 62/133] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:14:58 2022] PUT /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3/users/ddf38f7f3d7248769f04c04a1b75ce6c/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:14:58.998109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61ca86-136b-465c-9862-7d1b798984ab None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:59.092578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:59.235466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca25573-d22f-4fcf-b1f2-a454f3c312e7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:59.253469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61ca86-136b-465c-9862-7d1b798984ab None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:59.296448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:59.299345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ff80-7743-4f7a-b515-66901d41938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:59.300217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ff80-7743-4f7a-b515-66901d41938d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:59.300591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ff80-7743-4f7a-b515-66901d41938d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:59.301243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ff80-7743-4f7a-b515-66901d41938d None admin] PATH_INFO: `/v3/projects/8f143e4d2e324cd688773436ad8d7a49` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:59.302750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92ff80-7743-4f7a-b515-66901d41938d None admin] RBAC: Authorizing `identity:get_project(project_id=8f143e4d2e324cd688773436ad8d7a49)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:59.302924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43656b39-41c1-40e7-9fcb-eafd0c411cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:59.303529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656b39-41c1-40e7-9fcb-eafd0c411cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:59.303868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656b39-41c1-40e7-9fcb-eafd0c411cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:59.304185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43656b39-41c1-40e7-9fcb-eafd0c411cc0 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:59.304410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92ff80-7743-4f7a-b515-66901d41938d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:59.305352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43656b39-41c1-40e7-9fcb-eafd0c411cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:59.306237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 63/134] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:58 2022] GET /identity/v3/projects/8f143e4d2e324cd688773436ad8d7a49 => generated 312 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:59.307127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43656b39-41c1-40e7-9fcb-eafd0c411cc0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:59.307367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162db348-4c66-4773-aefd-44f5b7815096 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:59.307647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162db348-4c66-4773-aefd-44f5b7815096 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:59.307916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162db348-4c66-4773-aefd-44f5b7815096 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:59.309132 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 64/135] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:14:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:14:59.309642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 72/136] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:14:58 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:59.314932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c334b-2576-4f66-b6fa-37e8806ba0a1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:59.315215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c334b-2576-4f66-b6fa-37e8806ba0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:59.315490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c334b-2576-4f66-b6fa-37e8806ba0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:59.319640 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43656b39-41c1-40e7-9fcb-eafd0c411cc0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:59.341143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-246c334b-2576-4f66-b6fa-37e8806ba0a1 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:14:59.342343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-246c334b-2576-4f66-b6fa-37e8806ba0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:14:59.432986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 65/137] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:14:59.433745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c334b-2576-4f66-b6fa-37e8806ba0a1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:59.456553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43656b39-41c1-40e7-9fcb-eafd0c411cc0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:59.459244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:59.504903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:59.506127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:59.506500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:59.506778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:59.507518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] RBAC: Adding query filter params (role.id=a880da5b6bd94a89b2b2658b51f69ee9, scope.project.id=d38685d4e3e64d9ba662890b9147cac3, user.id=7b69638f9914428284970a3438274cfa) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:14:59.507827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:59.509314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:59.526540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 73/138] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:14:59 2022] GET /identity/v3/role_assignments?user.id=7b69638f9914428284970a3438274cfa&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=a880da5b6bd94a89b2b2658b51f69ee9 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:14:59.527450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:59.644055 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c334b-2576-4f66-b6fa-37e8806ba0a1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:59.690813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75ee7-be1b-45bc-a48f-c09bf8583379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:59.691418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75ee7-be1b-45bc-a48f-c09bf8583379 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:59.691701 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75ee7-be1b-45bc-a48f-c09bf8583379 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:59.691981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e75ee7-be1b-45bc-a48f-c09bf8583379 None admin] PATH_INFO: `/v3/projects/8f143e4d2e324cd688773436ad8d7a49/users/0898fbca99fc478e9cab823c368a904f/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:59.692532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:59.705165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e75ee7-be1b-45bc-a48f-c09bf8583379 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f143e4d2e324cd688773436ad8d7a49, user_id=0898fbca99fc478e9cab823c368a904f, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:14:59.706669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e75ee7-be1b-45bc-a48f-c09bf8583379 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:14:59.735231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 66/139] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:14:59 2022] PUT /identity/v3/projects/8f143e4d2e324cd688773436ad8d7a49/users/0898fbca99fc478e9cab823c368a904f/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:14:59.764736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5805221-b170-4658-a485-8b20e673fc60 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:14:59.811654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848cecd-b96f-4bd6-a7ab-96b13900d500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:14:59.812329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848cecd-b96f-4bd6-a7ab-96b13900d500 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:14:59.812615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848cecd-b96f-4bd6-a7ab-96b13900d500 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:14:59.812892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848cecd-b96f-4bd6-a7ab-96b13900d500 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:14:59.829796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848cecd-b96f-4bd6-a7ab-96b13900d500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-a848cecd-b96f-4bd6-a7ab-96b13900d500 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:14:59.836375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:14:59.838641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:14:59.840750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 74/140] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:14:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:14:59.845992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75ee7-be1b-45bc-a48f-c09bf8583379 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:14:59.953454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:14:59.980045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e75ee7-be1b-45bc-a48f-c09bf8583379 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:00.019670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:00.020332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:00.020649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:00.020942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:00.021622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:00.021930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:00.023327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:00.037248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 67/141] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:14:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:00.042654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848cecd-b96f-4bd6-a7ab-96b13900d500 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:00.165067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848cecd-b96f-4bd6-a7ab-96b13900d500 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:00.192526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:00.211774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad46d5e-cf70-4535-917e-a39d428a497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:00.212383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46d5e-cf70-4535-917e-a39d428a497b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:00.212665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46d5e-cf70-4535-917e-a39d428a497b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:00.212938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46d5e-cf70-4535-917e-a39d428a497b None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:00.214107 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad46d5e-cf70-4535-917e-a39d428a497b None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:00.215567 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad46d5e-cf70-4535-917e-a39d428a497b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:00.217517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 75/142] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:00 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:00.222676 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:00.350771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ba00-1a58-484d-8112-8d80ca8eee12 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:00.393047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575db859-8ede-47b0-b237-f0848e4a92af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:00.393669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575db859-8ede-47b0-b237-f0848e4a92af None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:00.393954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575db859-8ede-47b0-b237-f0848e4a92af None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:00.394231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575db859-8ede-47b0-b237-f0848e4a92af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:00.440487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:00.484162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575db859-8ede-47b0-b237-f0848e4a92af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:00.485579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575db859-8ede-47b0-b237-f0848e4a92af None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:00.609439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 68/143] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:00.616002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad46d5e-cf70-4535-917e-a39d428a497b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:00.698779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8253191-35c6-4b50-b500-a4dcf953ef1b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:00.699128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8253191-35c6-4b50-b500-a4dcf953ef1b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:00.699340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8253191-35c6-4b50-b500-a4dcf953ef1b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:00.701078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 69/144] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:00.706052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa2cbb-be80-4cce-aeb5-ab2382e2a58d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:00.706270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa2cbb-be80-4cce-aeb5-ab2382e2a58d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:00.706548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa2cbb-be80-4cce-aeb5-ab2382e2a58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:00.737719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0afa2cbb-be80-4cce-aeb5-ab2382e2a58d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:00.738849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0afa2cbb-be80-4cce-aeb5-ab2382e2a58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:00.802569 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad46d5e-cf70-4535-917e-a39d428a497b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:00.828129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:00.895733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 70/145] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:00.928129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cdb2a-f29e-4c4b-b93a-99e27b19966e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:00.928129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cdb2a-f29e-4c4b-b93a-99e27b19966e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:00.928129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cdb2a-f29e-4c4b-b93a-99e27b19966e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:00.928129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cdb2a-f29e-4c4b-b93a-99e27b19966e None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:00.936126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cdb2a-f29e-4c4b-b93a-99e27b19966e None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:00.936126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cdb2a-f29e-4c4b-b93a-99e27b19966e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:00.936449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 76/146] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:00 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:00.941681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f063dd-eb85-4d52-9e76-4d858e238e15 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:00.941987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f063dd-eb85-4d52-9e76-4d858e238e15 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:00.942281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f063dd-eb85-4d52-9e76-4d858e238e15 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:00.944623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 71/147] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:00.945572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cdb2a-f29e-4c4b-b93a-99e27b19966e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:00.961547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e7738-de96-4dbc-9803-3490f6886e22 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:00.961850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e7738-de96-4dbc-9803-3490f6886e22 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:00.962460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e7738-de96-4dbc-9803-3490f6886e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.011417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fa8e7738-de96-4dbc-9803-3490f6886e22 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:01.012564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fa8e7738-de96-4dbc-9803-3490f6886e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:01.064122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:01.122817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cdb2a-f29e-4c4b-b93a-99e27b19966e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:01.168916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:01.169532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:01.169809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:01.170078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.170801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] RBAC: Adding query filter params (role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=d38685d4e3e64d9ba662890b9147cac3, user.id=0898fbca99fc478e9cab823c368a904f) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:01.171105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:01.172573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:01.179496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 77/148] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:00 2022] GET /identity/v3/role_assignments?user.id=0898fbca99fc478e9cab823c368a904f&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:01.180932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:01.197718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 72/149] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:01.241233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e7738-de96-4dbc-9803-3490f6886e22 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:01.367095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08701111-9da7-4c71-98b3-5c2a5a610027 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:01.412731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:01.430609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9b316-0ed7-4f7d-b493-2d3cbdc2ec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:01.431722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9b316-0ed7-4f7d-b493-2d3cbdc2ec6b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:01.432439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9b316-0ed7-4f7d-b493-2d3cbdc2ec6b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:01.432918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9b316-0ed7-4f7d-b493-2d3cbdc2ec6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.438604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9b316-0ed7-4f7d-b493-2d3cbdc2ec6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:01.440381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9b316-0ed7-4f7d-b493-2d3cbdc2ec6b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:01.448027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 78/150] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:01.451812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e7738-de96-4dbc-9803-3490f6886e22 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:01.461800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9b316-0ed7-4f7d-b493-2d3cbdc2ec6b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:01.462547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:01.537520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a75e5-104e-4a7d-9e1a-619776218a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:01.539512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a75e5-104e-4a7d-9e1a-619776218a72 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:01.540013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a75e5-104e-4a7d-9e1a-619776218a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:01.540565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a75e5-104e-4a7d-9e1a-619776218a72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.553512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a75e5-104e-4a7d-9e1a-619776218a72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e77a75e5-104e-4a7d-9e1a-619776218a72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:01.559657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:01.560776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:01.561573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:01.564029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 73/151] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:01.571646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a75e5-104e-4a7d-9e1a-619776218a72 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:01.665483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9b316-0ed7-4f7d-b493-2d3cbdc2ec6b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:01.709710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a75e5-104e-4a7d-9e1a-619776218a72 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:01.722506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c01d77-aa40-4909-b561-afa0d1f94fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:01.723496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c01d77-aa40-4909-b561-afa0d1f94fc6 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:01.724488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c01d77-aa40-4909-b561-afa0d1f94fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:01.725066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c01d77-aa40-4909-b561-afa0d1f94fc6 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.725352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c01d77-aa40-4909-b561-afa0d1f94fc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:01.727495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c01d77-aa40-4909-b561-afa0d1f94fc6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:01.766030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 79/152] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:01 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:15:01.767165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda1de0-aa44-4381-8ffc-507c7af775b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:01.768618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda1de0-aa44-4381-8ffc-507c7af775b4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:01.769124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda1de0-aa44-4381-8ffc-507c7af775b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:01.769621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda1de0-aa44-4381-8ffc-507c7af775b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.770623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda1de0-aa44-4381-8ffc-507c7af775b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:01.771810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda1de0-aa44-4381-8ffc-507c7af775b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:01.773688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda1de0-aa44-4381-8ffc-507c7af775b4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:01.784034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 74/153] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:01.789793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c01d77-aa40-4909-b561-afa0d1f94fc6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:01.857636 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:01.921660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c01d77-aa40-4909-b561-afa0d1f94fc6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:01.964323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beb603-d7fe-4d64-8a27-219d230cd3c4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:01.964928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beb603-d7fe-4d64-8a27-219d230cd3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:01.965418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beb603-d7fe-4d64-8a27-219d230cd3c4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.967006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 75/154] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:01.972398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc53138-f289-48d7-9272-52e217e6f065 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:01.972658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc53138-f289-48d7-9272-52e217e6f065 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:01.972892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc53138-f289-48d7-9272-52e217e6f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.974966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e64f-6162-43c1-a241-a0a8308f151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:01.975981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e64f-6162-43c1-a241-a0a8308f151a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:01.976510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e64f-6162-43c1-a241-a0a8308f151a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:01.977002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e64f-6162-43c1-a241-a0a8308f151a None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:01.978523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64e64f-6162-43c1-a241-a0a8308f151a None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:01.980336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64e64f-6162-43c1-a241-a0a8308f151a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:01.982415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 80/155] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:01 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:01.988210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e64f-6162-43c1-a241-a0a8308f151a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:02.012410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4bc53138-f289-48d7-9272-52e217e6f065 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:02.014063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4bc53138-f289-48d7-9272-52e217e6f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:02.052852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:02.123159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 76/156] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:02.130852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e64f-6162-43c1-a241-a0a8308f151a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:02.147398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc53138-f289-48d7-9272-52e217e6f065 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:02.157814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:02.169042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a665c4-2086-432b-80f1-f53a8cf94256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:02.169764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a665c4-2086-432b-80f1-f53a8cf94256 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.170047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a665c4-2086-432b-80f1-f53a8cf94256 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.170320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a665c4-2086-432b-80f1-f53a8cf94256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:02.246263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a665c4-2086-432b-80f1-f53a8cf94256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:02.247854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a665c4-2086-432b-80f1-f53a8cf94256 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:02.262151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc53138-f289-48d7-9272-52e217e6f065 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:02.325857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dad07-e38a-455a-8bba-939eda98cc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:02.326890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dad07-e38a-455a-8bba-939eda98cc57 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.327463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dad07-e38a-455a-8bba-939eda98cc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.327932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73dad07-e38a-455a-8bba-939eda98cc57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:02.344038 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73dad07-e38a-455a-8bba-939eda98cc57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e73dad07-e38a-455a-8bba-939eda98cc57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:02.351240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:02.352230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:02.353013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:02.354220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 77/157] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:02.361108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dad07-e38a-455a-8bba-939eda98cc57 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:02.363882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 81/158] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:02.369835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a665c4-2086-432b-80f1-f53a8cf94256 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:02.392921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:02.397897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:02.493913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a665c4-2086-432b-80f1-f53a8cf94256 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:02.522358 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73dad07-e38a-455a-8bba-939eda98cc57 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:02.531518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4c301-3b8a-4190-b69f-828bee90ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:02.532575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4c301-3b8a-4190-b69f-828bee90ae11 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.533060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4c301-3b8a-4190-b69f-828bee90ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.533526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4c301-3b8a-4190-b69f-828bee90ae11 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:02.534945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4c301-3b8a-4190-b69f-828bee90ae11 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:02.536673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4c301-3b8a-4190-b69f-828bee90ae11 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:02.538550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 82/159] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:02 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:02.545254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4c301-3b8a-4190-b69f-828bee90ae11 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:02.579437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:02.580458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.581117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.581628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:02.582820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:02.583396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:02.587466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:02.599900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 78/160] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:02.607433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:02.661770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:02.681578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4c301-3b8a-4190-b69f-828bee90ae11 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:02.706954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:02.724381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:02.725080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.725512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.725845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:02.726797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] RBAC: Adding query filter params (role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=d38685d4e3e64d9ba662890b9147cac3, user.id=0898fbca99fc478e9cab823c368a904f) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:02.727117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:02.728642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:02.734964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 83/161] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:02 2022] GET /identity/v3/role_assignments?user.id=0898fbca99fc478e9cab823c368a904f&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:02.736822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d397f3e-d854-471f-b245-fa8f59ced833 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:02.777005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebe1bc-9b82-47f5-b60d-1826f7f1a325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:02.778432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe1bc-9b82-47f5-b60d-1826f7f1a325 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.778880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe1bc-9b82-47f5-b60d-1826f7f1a325 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.779341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe1bc-9b82-47f5-b60d-1826f7f1a325 None admin] PATH_INFO: `/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:02.780799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebe1bc-9b82-47f5-b60d-1826f7f1a325 None admin] RBAC: Authorizing `identity:get_user(user_id=ddf38f7f3d7248769f04c04a1b75ce6c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:02.782436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebe1bc-9b82-47f5-b60d-1826f7f1a325 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:02.784285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 79/162] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:02 2022] GET /identity/v3/users/ddf38f7f3d7248769f04c04a1b75ce6c => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:02.801648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:02.914615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8235d7-d6d7-4bb3-88ed-097218ed6386 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.915176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8235d7-d6d7-4bb3-88ed-097218ed6386 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.915623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8235d7-d6d7-4bb3-88ed-097218ed6386 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:02.917261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 80/163] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:02.922751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af4758-2c88-4f65-b6f0-9161039ebdf2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.923731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af4758-2c88-4f65-b6f0-9161039ebdf2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.924423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af4758-2c88-4f65-b6f0-9161039ebdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:02.951217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796cefea-9659-47dc-9766-637e7ceb2846 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:02.953475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-22af4758-2c88-4f65-b6f0-9161039ebdf2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:02.954930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-22af4758-2c88-4f65-b6f0-9161039ebdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:02.996339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40187c3-29b4-4494-8ff1-fb813c80de09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:02.996970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40187c3-29b4-4494-8ff1-fb813c80de09 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:02.997242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40187c3-29b4-4494-8ff1-fb813c80de09 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:02.997514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40187c3-29b4-4494-8ff1-fb813c80de09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.019960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:03.065446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 81/164] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:03.066919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35afa9-fa79-4580-bf9d-977ca6829b5b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:03.068018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35afa9-fa79-4580-bf9d-977ca6829b5b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:03.068458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35afa9-fa79-4580-bf9d-977ca6829b5b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.072131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 82/165] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:03.076724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b6ef8-11e2-4faa-927a-44fc492a430a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:03.077427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b6ef8-11e2-4faa-927a-44fc492a430a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:03.077844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b6ef8-11e2-4faa-927a-44fc492a430a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.094649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40187c3-29b4-4494-8ff1-fb813c80de09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:03.097202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40187c3-29b4-4494-8ff1-fb813c80de09 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:03.101239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-760b6ef8-11e2-4faa-927a-44fc492a430a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:03.102583 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-760b6ef8-11e2-4faa-927a-44fc492a430a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:03.187483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 83/166] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:03.188779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b6ef8-11e2-4faa-927a-44fc492a430a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:03.195095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:03.233737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 84/167] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:03.234885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40187c3-29b4-4494-8ff1-fb813c80de09 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:03.307366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b6ef8-11e2-4faa-927a-44fc492a430a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:03.320893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:03.349399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc8471-9fc0-479c-9190-ffa15652838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:03.350188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8471-9fc0-479c-9190-ffa15652838a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:03.350562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8471-9fc0-479c-9190-ffa15652838a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:03.350927 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc8471-9fc0-479c-9190-ffa15652838a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.351794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40187c3-29b4-4494-8ff1-fb813c80de09 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:03.366080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc8471-9fc0-479c-9190-ffa15652838a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d4fc8471-9fc0-479c-9190-ffa15652838a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:03.373786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:03.375144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 84/168] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:03.376165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc8471-9fc0-479c-9190-ffa15652838a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:03.417834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc2bed-dc6b-4561-80d5-7d34b34a9813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:03.418714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2bed-dc6b-4561-80d5-7d34b34a9813 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:03.419163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2bed-dc6b-4561-80d5-7d34b34a9813 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:03.419625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc2bed-dc6b-4561-80d5-7d34b34a9813 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.436251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc2bed-dc6b-4561-80d5-7d34b34a9813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-92cc2bed-dc6b-4561-80d5-7d34b34a9813 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:03.449939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:03.450882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:03.451746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:03.454421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 85/169] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:03.455391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc2bed-dc6b-4561-80d5-7d34b34a9813 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:03.472399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:03.492510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc8471-9fc0-479c-9190-ffa15652838a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:03.566794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d6228-1220-439b-b94c-11a6481cbe31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:03.567502 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6228-1220-439b-b94c-11a6481cbe31 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:03.567987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6228-1220-439b-b94c-11a6481cbe31 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:03.568287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6228-1220-439b-b94c-11a6481cbe31 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.571654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc2bed-dc6b-4561-80d5-7d34b34a9813 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:03.572425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d6228-1220-439b-b94c-11a6481cbe31 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:03.574105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d6228-1220-439b-b94c-11a6481cbe31 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:03.575782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 85/170] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:03 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:03.576573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d6228-1220-439b-b94c-11a6481cbe31 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:03.581459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:03.620967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:03.621874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:03.622469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:03.622930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.623843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:03.624359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:03.626030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:03.643356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 86/171] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:03.644439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:03.709429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d6228-1220-439b-b94c-11a6481cbe31 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:03.755892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:03.756543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:03.756826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:03.757101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.760547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:03.760860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:03.762255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:03.783261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:03.783864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 86/172] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:03.785160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:03.825962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739277aa-5b96-4b44-b08a-1693940d2fb0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:03.860559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:03.861331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:03.861617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:03.861889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:03.862622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=d38685d4e3e64d9ba662890b9147cac3, user.id=ddf38f7f3d7248769f04c04a1b75ce6c) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:03.862985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:03.865151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:03.871666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 87/173] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:03 2022] GET /identity/v3/role_assignments?user.id=ddf38f7f3d7248769f04c04a1b75ce6c&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=29fd031edd2b4f65aed8942853150f92 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:03.872551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:03.910465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:03.986349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1f633-2162-4015-aef3-34e6ba83e6e3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:04.025880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf4e4e-076f-43b2-ba62-df452f664f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:04.026529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf4e4e-076f-43b2-ba62-df452f664f96 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:04.026806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf4e4e-076f-43b2-ba62-df452f664f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:04.027073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf4e4e-076f-43b2-ba62-df452f664f96 None admin] PATH_INFO: `/v3/users/0898fbca99fc478e9cab823c368a904f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:04.028345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf4e4e-076f-43b2-ba62-df452f664f96 None admin] RBAC: Authorizing `identity:get_user(user_id=0898fbca99fc478e9cab823c368a904f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:04.029814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf4e4e-076f-43b2-ba62-df452f664f96 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:04.031512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 88/174] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:03 2022] GET /identity/v3/users/0898fbca99fc478e9cab823c368a904f => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:04.033582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b17008-eea8-4ef9-8a8f-699faacc3f0d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:04.047221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf4e4e-076f-43b2-ba62-df452f664f96 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:04.157492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40c03d-6721-4d42-9b32-7d9f3e21085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:04.158070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c03d-6721-4d42-9b32-7d9f3e21085d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:04.158357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c03d-6721-4d42-9b32-7d9f3e21085d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:04.158629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40c03d-6721-4d42-9b32-7d9f3e21085d None admin] PATH_INFO: `/v3/users/7b69638f9914428284970a3438274cfa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:04.164811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40c03d-6721-4d42-9b32-7d9f3e21085d None admin] RBAC: Authorizing `identity:get_user(user_id=7b69638f9914428284970a3438274cfa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:04.166277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40c03d-6721-4d42-9b32-7d9f3e21085d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:04.167888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf4e4e-076f-43b2-ba62-df452f664f96 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:04.168222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 87/175] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:03 2022] GET /identity/v3/users/7b69638f9914428284970a3438274cfa => generated 289 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:04.172120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:04.185159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40c03d-6721-4d42-9b32-7d9f3e21085d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:04.222470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4b1ec-c2ca-44f5-94a3-9e12f2ed0869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:04.223441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b1ec-c2ca-44f5-94a3-9e12f2ed0869 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:04.225323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b1ec-c2ca-44f5-94a3-9e12f2ed0869 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:04.225814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4b1ec-c2ca-44f5-94a3-9e12f2ed0869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:04.311822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4b1ec-c2ca-44f5-94a3-9e12f2ed0869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:04.313976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e4b1ec-c2ca-44f5-94a3-9e12f2ed0869 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:04.331530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:04.426332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40c03d-6721-4d42-9b32-7d9f3e21085d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:04.429197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 89/176] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:04.436979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4b1ec-c2ca-44f5-94a3-9e12f2ed0869 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:04.440385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:04.492667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2422fab-8850-4ae0-9b0f-68392c9764d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:04.493306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422fab-8850-4ae0-9b0f-68392c9764d7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:04.493587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422fab-8850-4ae0-9b0f-68392c9764d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:04.493857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2422fab-8850-4ae0-9b0f-68392c9764d7 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:04.495143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2422fab-8850-4ae0-9b0f-68392c9764d7 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:04.496634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2422fab-8850-4ae0-9b0f-68392c9764d7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:04.498439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 88/177] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:04 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:04.505787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2422fab-8850-4ae0-9b0f-68392c9764d7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:04.615070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4b1ec-c2ca-44f5-94a3-9e12f2ed0869 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:04.683882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87d323-73d9-4c7c-8839-6fec5b96516b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:04.684486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87d323-73d9-4c7c-8839-6fec5b96516b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:04.684766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87d323-73d9-4c7c-8839-6fec5b96516b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:04.685039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87d323-73d9-4c7c-8839-6fec5b96516b None admin] PATH_INFO: `/v3/projects/8f143e4d2e324cd688773436ad8d7a49` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:04.687307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87d323-73d9-4c7c-8839-6fec5b96516b None admin] RBAC: Authorizing `identity:get_project(project_id=8f143e4d2e324cd688773436ad8d7a49)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:04.689141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87d323-73d9-4c7c-8839-6fec5b96516b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:04.690996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2422fab-8850-4ae0-9b0f-68392c9764d7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:04.692194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 90/178] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:04 2022] GET /identity/v3/projects/8f143e4d2e324cd688773436ad8d7a49 => generated 312 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:04.696630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87d323-73d9-4c7c-8839-6fec5b96516b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:04.747223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:04.754002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a819664-fe06-49a4-b75f-c56544ac6763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:04.754682 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a819664-fe06-49a4-b75f-c56544ac6763 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:04.755853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a819664-fe06-49a4-b75f-c56544ac6763 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:04.756374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a819664-fe06-49a4-b75f-c56544ac6763 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3/users/7b69638f9914428284970a3438274cfa/roles/a880da5b6bd94a89b2b2658b51f69ee9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:04.761590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a819664-fe06-49a4-b75f-c56544ac6763 None admin] RBAC: Authorizing `identity:create_grant(project_id=d38685d4e3e64d9ba662890b9147cac3, user_id=7b69638f9914428284970a3438274cfa, role_id=a880da5b6bd94a89b2b2658b51f69ee9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:04.765010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a819664-fe06-49a4-b75f-c56544ac6763 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:04.788922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 89/179] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:04 2022] PUT /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3/users/7b69638f9914428284970a3438274cfa/roles/a880da5b6bd94a89b2b2658b51f69ee9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:04.824604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:04.863764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87d323-73d9-4c7c-8839-6fec5b96516b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:04.913435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3b904-c429-4b7e-b80e-6c2aad7eea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:04.914062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3b904-c429-4b7e-b80e-6c2aad7eea9a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:04.914345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3b904-c429-4b7e-b80e-6c2aad7eea9a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:04.914619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3b904-c429-4b7e-b80e-6c2aad7eea9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:04.915366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3b904-c429-4b7e-b80e-6c2aad7eea9a None admin] RBAC: Adding query filter params (role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=8f143e4d2e324cd688773436ad8d7a49, user.id=0898fbca99fc478e9cab823c368a904f) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:04.915733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3b904-c429-4b7e-b80e-6c2aad7eea9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:04.917294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3b904-c429-4b7e-b80e-6c2aad7eea9a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:04.930074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 91/180] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:04 2022] GET /identity/v3/role_assignments?user.id=0898fbca99fc478e9cab823c368a904f&scope.project.id=8f143e4d2e324cd688773436ad8d7a49&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:05.064502 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7d404-fa3b-42eb-aeeb-cf4f8bd4436f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.064789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7d404-fa3b-42eb-aeeb-cf4f8bd4436f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.065135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7d404-fa3b-42eb-aeeb-cf4f8bd4436f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.066689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 90/181] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:05.073120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476e7a-0c41-42a5-ad22-58eeea7a5bdb None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.074077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476e7a-0c41-42a5-ad22-58eeea7a5bdb None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.074579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa476e7a-0c41-42a5-ad22-58eeea7a5bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.095569 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-fa476e7a-0c41-42a5-ad22-58eeea7a5bdb None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:05.097168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fa476e7a-0c41-42a5-ad22-58eeea7a5bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:05.169951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 92/182] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:05.195719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7d404-fa3b-42eb-aeeb-cf4f8bd4436f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:05.314886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7d404-fa3b-42eb-aeeb-cf4f8bd4436f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:05.357657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471102a-6bef-498e-984e-4fd9679b4d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:05.358638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471102a-6bef-498e-984e-4fd9679b4d39 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.359175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471102a-6bef-498e-984e-4fd9679b4d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.359719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471102a-6bef-498e-984e-4fd9679b4d39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.373732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471102a-6bef-498e-984e-4fd9679b4d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2471102a-6bef-498e-984e-4fd9679b4d39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:05.379118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:05.380575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:05.381363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:05.391007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa476e7a-0c41-42a5-ad22-58eeea7a5bdb None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:05.394530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 91/183] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:05.454692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968903f1-e27a-4d64-8691-1bacfde9a71f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.455014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968903f1-e27a-4d64-8691-1bacfde9a71f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.455260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968903f1-e27a-4d64-8691-1bacfde9a71f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.456641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 92/184] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:05.464888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f566e-9ceb-4052-9d73-101794de9e6d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.465190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f566e-9ceb-4052-9d73-101794de9e6d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.465436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f566e-9ceb-4052-9d73-101794de9e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.469955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:05.486599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-070f566e-9ceb-4052-9d73-101794de9e6d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:05.488559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-070f566e-9ceb-4052-9d73-101794de9e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:05.512431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa476e7a-0c41-42a5-ad22-58eeea7a5bdb None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:05.549653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:05.555231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:05.556366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.556861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.557343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.558209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:05.558921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:05.560520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:05.570091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 93/185] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:05.574302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 93/186] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:05.575822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:05.597912 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f566e-9ceb-4052-9d73-101794de9e6d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:05.707788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040ea14-c2d0-4baf-8471-b9c28e3280f3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:05.728013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f566e-9ceb-4052-9d73-101794de9e6d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:05.736496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:05.747876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681d47d-19c9-40eb-a712-61f355ed79e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:05.748543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681d47d-19c9-40eb-a712-61f355ed79e0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.748821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681d47d-19c9-40eb-a712-61f355ed79e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.749094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681d47d-19c9-40eb-a712-61f355ed79e0 None admin] PATH_INFO: `/v3/users/aa0dc6ff504e459b84e130def4df94a6` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.764510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681d47d-19c9-40eb-a712-61f355ed79e0 None admin] RBAC: Authorizing `identity:get_user(user_id=aa0dc6ff504e459b84e130def4df94a6)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:05.765971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681d47d-19c9-40eb-a712-61f355ed79e0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:05.766354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3cde1e-bd32-433e-9c21-c655cb1694eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:05.767224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cde1e-bd32-433e-9c21-c655cb1694eb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.767882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 94/187] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:05 2022] GET /identity/v3/users/aa0dc6ff504e459b84e130def4df94a6 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:05.768228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cde1e-bd32-433e-9c21-c655cb1694eb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.768734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3cde1e-bd32-433e-9c21-c655cb1694eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.773506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681d47d-19c9-40eb-a712-61f355ed79e0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:05.780636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3cde1e-bd32-433e-9c21-c655cb1694eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5e3cde1e-bd32-433e-9c21-c655cb1694eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:05.785364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:05.786578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:05.787559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:05.791353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 94/188] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:05.792487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d69f59-4a90-4024-83cc-742340233b12 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.792767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d69f59-4a90-4024-83cc-742340233b12 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.793091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d69f59-4a90-4024-83cc-742340233b12 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.794475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 95/189] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:05.796208 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d69f59-4a90-4024-83cc-742340233b12 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:05.841780 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:05.883411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681d47d-19c9-40eb-a712-61f355ed79e0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:05.905594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d69f59-4a90-4024-83cc-742340233b12 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:05.918851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7586694-0b77-436d-839d-c4df5fa9a029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:05.919570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586694-0b77-436d-839d-c4df5fa9a029 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.919849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586694-0b77-436d-839d-c4df5fa9a029 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.920265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7586694-0b77-436d-839d-c4df5fa9a029 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.940372 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1660e5d-40af-4496-9edb-d50c18d5c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:05.940977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1660e5d-40af-4496-9edb-d50c18d5c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.941252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1660e5d-40af-4496-9edb-d50c18d5c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.941519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1660e5d-40af-4496-9edb-d50c18d5c1a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.942226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1660e5d-40af-4496-9edb-d50c18d5c1a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:05.943276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1660e5d-40af-4496-9edb-d50c18d5c1a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:05.944688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1660e5d-40af-4496-9edb-d50c18d5c1a3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:05.956660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 96/190] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:05.956660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10693993-372b-4928-a5f5-6219e66d3c4c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:05.956660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10693993-372b-4928-a5f5-6219e66d3c4c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:05.956660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10693993-372b-4928-a5f5-6219e66d3c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:05.971147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-10693993-372b-4928-a5f5-6219e66d3c4c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:05.972370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-10693993-372b-4928-a5f5-6219e66d3c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:05.982315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:06.000223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7586694-0b77-436d-839d-c4df5fa9a029 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.001689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7586694-0b77-436d-839d-c4df5fa9a029 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.044022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 97/191] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:06.053182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10693993-372b-4928-a5f5-6219e66d3c4c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:06.085177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:06.116372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 95/192] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 16:15:06.117143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7586694-0b77-436d-839d-c4df5fa9a029 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:06.165430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10693993-372b-4928-a5f5-6219e66d3c4c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:06.200870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15795f-93a9-4527-aced-756a75138a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:06.201419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15795f-93a9-4527-aced-756a75138a1c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:06.201696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15795f-93a9-4527-aced-756a75138a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:06.201968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15795f-93a9-4527-aced-756a75138a1c None admin] PATH_INFO: `/v3/users/aa0dc6ff504e459b84e130def4df94a6` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:06.203104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15795f-93a9-4527-aced-756a75138a1c None admin] RBAC: Authorizing `identity:get_user(user_id=aa0dc6ff504e459b84e130def4df94a6)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.204610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15795f-93a9-4527-aced-756a75138a1c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.204721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:06.206442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 98/193] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:06 2022] GET /identity/v3/users/aa0dc6ff504e459b84e130def4df94a6 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:06.207246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15795f-93a9-4527-aced-756a75138a1c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:06.228487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7586694-0b77-436d-839d-c4df5fa9a029 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:06.264396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5b562-511c-46b9-b1a7-7e7f12ba5d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:06.265025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b562-511c-46b9-b1a7-7e7f12ba5d1b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:06.265303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b562-511c-46b9-b1a7-7e7f12ba5d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:06.265569 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b562-511c-46b9-b1a7-7e7f12ba5d1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:06.266675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5b562-511c-46b9-b1a7-7e7f12ba5d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.268122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5b562-511c-46b9-b1a7-7e7f12ba5d1b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.274330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 96/194] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:06.274330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5b562-511c-46b9-b1a7-7e7f12ba5d1b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:06.311675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:06.315637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15795f-93a9-4527-aced-756a75138a1c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:06.352079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6fb1b-c972-4b67-9350-641782c79e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:06.352830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6fb1b-c972-4b67-9350-641782c79e59 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:06.353176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6fb1b-c972-4b67-9350-641782c79e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:06.353450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6fb1b-c972-4b67-9350-641782c79e59 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:06.354566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6fb1b-c972-4b67-9350-641782c79e59 None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.355987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6fb1b-c972-4b67-9350-641782c79e59 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.362447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 99/195] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:06 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:06.362447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6fb1b-c972-4b67-9350-641782c79e59 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:06.381896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5b562-511c-46b9-b1a7-7e7f12ba5d1b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:06.418366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d14172-3bc4-4d59-b93f-43be69210ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:06.418964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d14172-3bc4-4d59-b93f-43be69210ffc None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:06.419241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d14172-3bc4-4d59-b93f-43be69210ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:06.419510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d14172-3bc4-4d59-b93f-43be69210ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:06.441840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:06.468728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6fb1b-c972-4b67-9350-641782c79e59 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:06.494026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d14172-3bc4-4d59-b93f-43be69210ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.495499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d14172-3bc4-4d59-b93f-43be69210ffc None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.505581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920797-7dd9-4568-9334-14439d3e03d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:06.506190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920797-7dd9-4568-9334-14439d3e03d2 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:06.506474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920797-7dd9-4568-9334-14439d3e03d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:06.506947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920797-7dd9-4568-9334-14439d3e03d2 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:06.507391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a920797-7dd9-4568-9334-14439d3e03d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.508854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a920797-7dd9-4568-9334-14439d3e03d2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.536644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 100/196] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:06 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:06.536644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920797-7dd9-4568-9334-14439d3e03d2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:06.570213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:06.630182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 97/197] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:06.636738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d14172-3bc4-4d59-b93f-43be69210ffc None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:06.659584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920797-7dd9-4568-9334-14439d3e03d2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:06.696621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:06.711366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:06.711994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:06.712603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:06.712882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:06.714311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=fa381b3d44664d6ab4beefcf506a841b, user.id=aa0dc6ff504e459b84e130def4df94a6) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:06.714636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.716475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.725332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 101/198] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:06 2022] GET /identity/v3/role_assignments?user.id=aa0dc6ff504e459b84e130def4df94a6&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=29fd031edd2b4f65aed8942853150f92 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:06.757633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d14172-3bc4-4d59-b93f-43be69210ffc None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:06.796870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edae52f-e3a8-4cb0-818a-02502b669a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:06.797550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edae52f-e3a8-4cb0-818a-02502b669a2b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:06.797866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edae52f-e3a8-4cb0-818a-02502b669a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:06.798201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edae52f-e3a8-4cb0-818a-02502b669a2b None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:06.799622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edae52f-e3a8-4cb0-818a-02502b669a2b None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.801184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edae52f-e3a8-4cb0-818a-02502b669a2b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.803325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 98/199] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:06 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:06.810133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:06.918741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:06.933145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5db07-7b3a-44cf-b417-30a3e31d5a8f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:06.973187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f388293-6c5f-48cf-a32e-f5fe38f3240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:06.974338 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f388293-6c5f-48cf-a32e-f5fe38f3240c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:06.974607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f388293-6c5f-48cf-a32e-f5fe38f3240c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:06.974870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f388293-6c5f-48cf-a32e-f5fe38f3240c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:06.975600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f388293-6c5f-48cf-a32e-f5fe38f3240c None admin] RBAC: Adding query filter params (role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=fa381b3d44664d6ab4beefcf506a841b, user.id=aa0dc6ff504e459b84e130def4df94a6) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:06.975900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f388293-6c5f-48cf-a32e-f5fe38f3240c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:06.977403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f388293-6c5f-48cf-a32e-f5fe38f3240c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:06.983302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 102/200] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:06 2022] GET /identity/v3/role_assignments?user.id=aa0dc6ff504e459b84e130def4df94a6&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:07.077444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069c161-b0fc-41a9-8a86-080b5e7bcade None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:07.077756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069c161-b0fc-41a9-8a86-080b5e7bcade None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:07.078066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069c161-b0fc-41a9-8a86-080b5e7bcade None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:07.079457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 99/201] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:07.086374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd96e9-1a89-4497-a83b-d64d9ddcc77f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:07.086654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd96e9-1a89-4497-a83b-d64d9ddcc77f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:07.086924 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd96e9-1a89-4497-a83b-d64d9ddcc77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:07.107017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-51cd96e9-1a89-4497-a83b-d64d9ddcc77f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:07.108925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-51cd96e9-1a89-4497-a83b-d64d9ddcc77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:07.201163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 103/202] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:07.235398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069c161-b0fc-41a9-8a86-080b5e7bcade None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:07.377849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069c161-b0fc-41a9-8a86-080b5e7bcade None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:07.407076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:07.423148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984578c5-c94f-43b0-a194-7d0f2653c561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:07.423795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984578c5-c94f-43b0-a194-7d0f2653c561 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:07.424121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984578c5-c94f-43b0-a194-7d0f2653c561 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:07.424430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984578c5-c94f-43b0-a194-7d0f2653c561 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:07.442009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984578c5-c94f-43b0-a194-7d0f2653c561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-984578c5-c94f-43b0-a194-7d0f2653c561 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:07.446913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:07.447804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:07.448734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:07.449360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 100/203] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:07.453929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd96e9-1a89-4497-a83b-d64d9ddcc77f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:07.573721 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:07.594490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd96e9-1a89-4497-a83b-d64d9ddcc77f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:07.648881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:07.649507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:07.649784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:07.650051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:07.650792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:07.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:07.652487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:07.661877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 104/204] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:07.666993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984578c5-c94f-43b0-a194-7d0f2653c561 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:07.799979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984578c5-c94f-43b0-a194-7d0f2653c561 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:07.851247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef495a8-26d2-4fd1-9cf1-42a847921f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:07.851901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef495a8-26d2-4fd1-9cf1-42a847921f8a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:07.852288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef495a8-26d2-4fd1-9cf1-42a847921f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:07.852626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef495a8-26d2-4fd1-9cf1-42a847921f8a None admin] PATH_INFO: `/v3/users/7b69638f9914428284970a3438274cfa` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:07.853840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef495a8-26d2-4fd1-9cf1-42a847921f8a None admin] RBAC: Authorizing `identity:get_user(user_id=7b69638f9914428284970a3438274cfa)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:07.855320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef495a8-26d2-4fd1-9cf1-42a847921f8a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:07.857122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 101/205] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:07 2022] GET /identity/v3/users/7b69638f9914428284970a3438274cfa => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:07.862526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:07.992229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b410a5-3aaf-478d-be7f-d61f53e4a29c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:08.031113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f039ae-f315-4a6d-b4e8-ad32f4ecfb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:08.032091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f039ae-f315-4a6d-b4e8-ad32f4ecfb89 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:08.032775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f039ae-f315-4a6d-b4e8-ad32f4ecfb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:08.033248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f039ae-f315-4a6d-b4e8-ad32f4ecfb89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:08.070062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:08.124003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f039ae-f315-4a6d-b4e8-ad32f4ecfb89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:08.125532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f039ae-f315-4a6d-b4e8-ad32f4ecfb89 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:08.248617 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 105/206] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:08.254433 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef495a8-26d2-4fd1-9cf1-42a847921f8a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:08.337374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:08.410753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef495a8-26d2-4fd1-9cf1-42a847921f8a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:08.451523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd807475-b74d-4ea4-af39-aa5cb6ac9078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:08.452564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd807475-b74d-4ea4-af39-aa5cb6ac9078 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:08.453070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd807475-b74d-4ea4-af39-aa5cb6ac9078 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:08.453597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd807475-b74d-4ea4-af39-aa5cb6ac9078 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:08.455001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd807475-b74d-4ea4-af39-aa5cb6ac9078 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:08.456687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd807475-b74d-4ea4-af39-aa5cb6ac9078 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:08.459141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 102/207] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:08 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:08.465491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f039ae-f315-4a6d-b4e8-ad32f4ecfb89 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:08.548461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:08.617539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f039ae-f315-4a6d-b4e8-ad32f4ecfb89 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:08.659515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157370c3-068e-447e-b61e-b10eabd09176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:08.660137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157370c3-068e-447e-b61e-b10eabd09176 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:08.660424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157370c3-068e-447e-b61e-b10eabd09176 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:08.660699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157370c3-068e-447e-b61e-b10eabd09176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:08.661491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157370c3-068e-447e-b61e-b10eabd09176 None admin] RBAC: Adding query filter params (role.id=a880da5b6bd94a89b2b2658b51f69ee9, scope.project.id=d38685d4e3e64d9ba662890b9147cac3, user.id=7b69638f9914428284970a3438274cfa) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:08.661799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157370c3-068e-447e-b61e-b10eabd09176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:08.663240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157370c3-068e-447e-b61e-b10eabd09176 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:08.669225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 106/208] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:08 2022] GET /identity/v3/role_assignments?user.id=7b69638f9914428284970a3438274cfa&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=a880da5b6bd94a89b2b2658b51f69ee9 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:09.267784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8c3e7-7778-4259-addd-73cd999c268a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.268829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8c3e7-7778-4259-addd-73cd999c268a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.269352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8c3e7-7778-4259-addd-73cd999c268a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.271544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 103/209] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:09.278203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0d2c6-13c9-41d9-8dfc-6a36cf38b774 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.279094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0d2c6-13c9-41d9-8dfc-6a36cf38b774 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.279557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0d2c6-13c9-41d9-8dfc-6a36cf38b774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.299188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-02d0d2c6-13c9-41d9-8dfc-6a36cf38b774 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:09.300386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-02d0d2c6-13c9-41d9-8dfc-6a36cf38b774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:09.324290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec7f7b-73f6-4427-a5cc-9682c9d3436c None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.324905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec7f7b-73f6-4427-a5cc-9682c9d3436c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.325403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec7f7b-73f6-4427-a5cc-9682c9d3436c None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.327164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 104/210] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:09.333006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505be722-ffec-4e1f-acf7-3ca56188919b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.333594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505be722-ffec-4e1f-acf7-3ca56188919b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.334097 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505be722-ffec-4e1f-acf7-3ca56188919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.356772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-505be722-ffec-4e1f-acf7-3ca56188919b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:09.358317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-505be722-ffec-4e1f-acf7-3ca56188919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:09.385357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 107/211] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:09.386706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9339f2e-c065-4adb-9fa5-892adb647aa2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.386997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9339f2e-c065-4adb-9fa5-892adb647aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.387270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9339f2e-c065-4adb-9fa5-892adb647aa2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.388782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 108/212] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:09.405379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952daf3-0e4d-4ae8-8144-20c437306ccc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.405655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952daf3-0e4d-4ae8-8144-20c437306ccc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.406480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952daf3-0e4d-4ae8-8144-20c437306ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.428044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3952daf3-0e4d-4ae8-8144-20c437306ccc None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:09.429247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3952daf3-0e4d-4ae8-8144-20c437306ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:09.434885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 105/213] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:09.435551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505be722-ffec-4e1f-acf7-3ca56188919b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:09.460502 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:09.484444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:09.498046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 109/214] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:09.504422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee628924-b3fd-4860-86d7-a187b459a3ac None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.504422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee628924-b3fd-4860-86d7-a187b459a3ac None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.504422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee628924-b3fd-4860-86d7-a187b459a3ac None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.504422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 110/215] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:09.504422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee628924-b3fd-4860-86d7-a187b459a3ac None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:09.542400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505be722-ffec-4e1f-acf7-3ca56188919b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:09.576276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f0332-de2b-45f3-8dd0-ae70d2e59c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:09.576858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0332-de2b-45f3-8dd0-ae70d2e59c01 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.577174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0332-de2b-45f3-8dd0-ae70d2e59c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.577477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0332-de2b-45f3-8dd0-ae70d2e59c01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.588186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f0332-de2b-45f3-8dd0-ae70d2e59c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-f70f0332-de2b-45f3-8dd0-ae70d2e59c01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:09.593385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:09.594369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:09.595189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 106/216] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:09.595847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24215ad7-0d9b-4886-a52c-fc3eaaa2c7c6 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.595913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24215ad7-0d9b-4886-a52c-fc3eaaa2c7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.596212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24215ad7-0d9b-4886-a52c-fc3eaaa2c7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.614871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-24215ad7-0d9b-4886-a52c-fc3eaaa2c7c6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:09.615695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee628924-b3fd-4860-86d7-a187b459a3ac None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:09.616234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-24215ad7-0d9b-4886-a52c-fc3eaaa2c7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:09.651097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0f2cc-7ef7-41e7-9304-168ce9527d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:09.651720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f2cc-7ef7-41e7-9304-168ce9527d24 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.651993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f2cc-7ef7-41e7-9304-168ce9527d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.652291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f2cc-7ef7-41e7-9304-168ce9527d24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.653395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0f2cc-7ef7-41e7-9304-168ce9527d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:09.654819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e0f2cc-7ef7-41e7-9304-168ce9527d24 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:09.656411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 111/217] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:09.662208 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0f2cc-7ef7-41e7-9304-168ce9527d24 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:09.683933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 107/218] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:09.685245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24215ad7-0d9b-4886-a52c-fc3eaaa2c7c6 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:09.708381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:09.721771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:09.761646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0f2cc-7ef7-41e7-9304-168ce9527d24 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:09.787365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24215ad7-0d9b-4886-a52c-fc3eaaa2c7c6 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:09.794477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80df86-1a97-43c7-8059-b8559175e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:09.795103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80df86-1a97-43c7-8059-b8559175e41c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.795383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80df86-1a97-43c7-8059-b8559175e41c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.795654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80df86-1a97-43c7-8059-b8559175e41c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.806037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80df86-1a97-43c7-8059-b8559175e41c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6f80df86-1a97-43c7-8059-b8559175e41c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:09.811216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:09.812196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:09.813010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 112/219] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:09.814932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80df86-1a97-43c7-8059-b8559175e41c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:09.820995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:09.821617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.821925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.822224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.822919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] RBAC: Adding query filter params (name=member) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:09.823250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:09.824680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:09.834219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 108/220] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:09.834793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:09.907091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:09.913536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80df86-1a97-43c7-8059-b8559175e41c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:09.931662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:09.935258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b7ba8-73ea-4bb0-9b40-691a5d6da8cc None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:09.946832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e56d-2c9b-44cc-bc3c-7834729fb65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:09.947430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e56d-2c9b-44cc-bc3c-7834729fb65c None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.947702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e56d-2c9b-44cc-bc3c-7834729fb65c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.947970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e56d-2c9b-44cc-bc3c-7834729fb65c None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.948635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7e56d-2c9b-44cc-bc3c-7834729fb65c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:09.950064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7e56d-2c9b-44cc-bc3c-7834729fb65c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:09.971083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 113/221] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:09 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:09.971256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e842f-7f93-4eec-aaf6-e62b8d88dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:09.971853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e842f-7f93-4eec-aaf6-e62b8d88dd65 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:09.971967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e56d-2c9b-44cc-bc3c-7834729fb65c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:09.972228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e842f-7f93-4eec-aaf6-e62b8d88dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:09.972574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e842f-7f93-4eec-aaf6-e62b8d88dd65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:09.992845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e842f-7f93-4eec-aaf6-e62b8d88dd65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-f14e842f-7f93-4eec-aaf6-e62b8d88dd65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:09.998401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:09.999416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:10.000258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 109/222] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:10.000944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e842f-7f93-4eec-aaf6-e62b8d88dd65 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.105726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e56d-2c9b-44cc-bc3c-7834729fb65c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.126657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e842f-7f93-4eec-aaf6-e62b8d88dd65 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.149708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.150323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.150604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.150868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.151532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:10.151829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.153322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.163944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 114/223] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:10.164673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.165220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95b5a4-3c77-4005-aade-ec60529b4157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.165867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b5a4-3c77-4005-aade-ec60529b4157 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.166144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b5a4-3c77-4005-aade-ec60529b4157 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.166412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b5a4-3c77-4005-aade-ec60529b4157 None admin] PATH_INFO: `/v3/users/c545c501ee3f4d048d5387a80900dfae` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.182318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:10.182696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95b5a4-3c77-4005-aade-ec60529b4157 None admin] RBAC: Authorizing `identity:get_user(user_id=c545c501ee3f4d048d5387a80900dfae)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.184197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95b5a4-3c77-4005-aade-ec60529b4157 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.186048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 110/224] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:09 2022] GET /identity/v3/users/c545c501ee3f4d048d5387a80900dfae => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:10.191634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95b5a4-3c77-4005-aade-ec60529b4157 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.214665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:10.293720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc5cc0-a038-4f39-9458-0182c251223e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.320566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95b5a4-3c77-4005-aade-ec60529b4157 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.343418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.344516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.344798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.345068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.345739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:10.346315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.347885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.360574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 115/225] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:10.362110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.369147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d03e6-0e7e-4a9c-b2d0-76ffa8bf14c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.369791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d03e6-0e7e-4a9c-b2d0-76ffa8bf14c1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.370069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d03e6-0e7e-4a9c-b2d0-76ffa8bf14c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.370395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d03e6-0e7e-4a9c-b2d0-76ffa8bf14c1 None admin] PATH_INFO: `/v3/users/aa0dc6ff504e459b84e130def4df94a6` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.372591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d03e6-0e7e-4a9c-b2d0-76ffa8bf14c1 None admin] RBAC: Authorizing `identity:get_user(user_id=aa0dc6ff504e459b84e130def4df94a6)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.373809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d03e6-0e7e-4a9c-b2d0-76ffa8bf14c1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.375598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 111/226] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:10 2022] GET /identity/v3/users/aa0dc6ff504e459b84e130def4df94a6 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:10.376312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d03e6-0e7e-4a9c-b2d0-76ffa8bf14c1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.473076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:10.481135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:10.493608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d03e6-0e7e-4a9c-b2d0-76ffa8bf14c1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.512808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bd9bd-0b72-4654-8595-a3c5bdae002d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.531946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e33b8-d9f9-4e3d-8ce0-fb6b5bdb33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.532573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e33b8-d9f9-4e3d-8ce0-fb6b5bdb33c9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.532899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e33b8-d9f9-4e3d-8ce0-fb6b5bdb33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.533180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e33b8-d9f9-4e3d-8ce0-fb6b5bdb33c9 None admin] PATH_INFO: `/v3/users/aa0dc6ff504e459b84e130def4df94a6` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.534344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e33b8-d9f9-4e3d-8ce0-fb6b5bdb33c9 None admin] RBAC: Authorizing `identity:get_user(user_id=aa0dc6ff504e459b84e130def4df94a6)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.535964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e33b8-d9f9-4e3d-8ce0-fb6b5bdb33c9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.538080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 112/227] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:10 2022] GET /identity/v3/users/aa0dc6ff504e459b84e130def4df94a6 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:10.539514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e33b8-d9f9-4e3d-8ce0-fb6b5bdb33c9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.564855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3141-7cd6-4da7-a149-6dbd13ff9ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.566375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3141-7cd6-4da7-a149-6dbd13ff9ddf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.566870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3141-7cd6-4da7-a149-6dbd13ff9ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.568180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3141-7cd6-4da7-a149-6dbd13ff9ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.654012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e33b8-d9f9-4e3d-8ce0-fb6b5bdb33c9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.659640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d3141-7cd6-4da7-a149-6dbd13ff9ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.661421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d3141-7cd6-4da7-a149-6dbd13ff9ddf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.690812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7e442-d4e7-4fc9-ab6c-8bd12f034bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.691476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e442-d4e7-4fc9-ab6c-8bd12f034bec None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.691757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e442-d4e7-4fc9-ab6c-8bd12f034bec None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.692587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7e442-d4e7-4fc9-ab6c-8bd12f034bec None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.694145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7e442-d4e7-4fc9-ab6c-8bd12f034bec None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.695437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7e442-d4e7-4fc9-ab6c-8bd12f034bec None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.701757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 113/228] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:10 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:10.701757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7e442-d4e7-4fc9-ab6c-8bd12f034bec None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.736516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:10.755051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:10.782760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 116/229] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:10.788439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3141-7cd6-4da7-a149-6dbd13ff9ddf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.829092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7e442-d4e7-4fc9-ab6c-8bd12f034bec None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.867397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec22de-ac13-4d7b-b287-48cd89415474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.868426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec22de-ac13-4d7b-b287-48cd89415474 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.868851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec22de-ac13-4d7b-b287-48cd89415474 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.869302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec22de-ac13-4d7b-b287-48cd89415474 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.870656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec22de-ac13-4d7b-b287-48cd89415474 None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.872591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec22de-ac13-4d7b-b287-48cd89415474 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.879489 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 114/230] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:10 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:10.881399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec22de-ac13-4d7b-b287-48cd89415474 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.908033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3141-7cd6-4da7-a149-6dbd13ff9ddf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:10.947726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfdd28-a65d-492f-99d9-df23609d23a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:10.948787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfdd28-a65d-492f-99d9-df23609d23a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:10.949243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfdd28-a65d-492f-99d9-df23609d23a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:10.949828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bfdd28-a65d-492f-99d9-df23609d23a1 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b/users/aa0dc6ff504e459b84e130def4df94a6/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:10.952127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfdd28-a65d-492f-99d9-df23609d23a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa381b3d44664d6ab4beefcf506a841b, user_id=aa0dc6ff504e459b84e130def4df94a6, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:10.953970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bfdd28-a65d-492f-99d9-df23609d23a1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:10.969930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 117/231] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:10 2022] PUT /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b/users/aa0dc6ff504e459b84e130def4df94a6/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:10.977653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfdd28-a65d-492f-99d9-df23609d23a1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:10.995722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:11.014564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec22de-ac13-4d7b-b287-48cd89415474 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:11.024421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:11.097604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1f675-9945-4912-92d8-f7ae9ac10144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:11.098191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1f675-9945-4912-92d8-f7ae9ac10144 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:11.098585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1f675-9945-4912-92d8-f7ae9ac10144 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:11.099026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1f675-9945-4912-92d8-f7ae9ac10144 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:11.104891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1f675-9945-4912-92d8-f7ae9ac10144 None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:11.106472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1f675-9945-4912-92d8-f7ae9ac10144 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:11.108296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 115/232] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:10 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:11.109422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bfdd28-a65d-492f-99d9-df23609d23a1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:11.115220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1f675-9945-4912-92d8-f7ae9ac10144 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:11.152853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9481d1-bec7-4949-9b59-f20972398206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:11.153835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9481d1-bec7-4949-9b59-f20972398206 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:11.154320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9481d1-bec7-4949-9b59-f20972398206 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:11.154803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9481d1-bec7-4949-9b59-f20972398206 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b/users/aa0dc6ff504e459b84e130def4df94a6/roles/3b5810bc1bb047f7851cef3afb74dcd0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:11.157480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9481d1-bec7-4949-9b59-f20972398206 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa381b3d44664d6ab4beefcf506a841b, user_id=aa0dc6ff504e459b84e130def4df94a6, role_id=3b5810bc1bb047f7851cef3afb74dcd0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:11.159370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9481d1-bec7-4949-9b59-f20972398206 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:11.191189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 118/233] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:10 2022] PUT /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b/users/aa0dc6ff504e459b84e130def4df94a6/roles/3b5810bc1bb047f7851cef3afb74dcd0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:11.273771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1f675-9945-4912-92d8-f7ae9ac10144 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:11.313610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b571b5-4582-45b3-840d-c69cb519a574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:11.314574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b571b5-4582-45b3-840d-c69cb519a574 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:11.315031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b571b5-4582-45b3-840d-c69cb519a574 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:11.315475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b571b5-4582-45b3-840d-c69cb519a574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:11.316445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b571b5-4582-45b3-840d-c69cb519a574 None admin] RBAC: Adding query filter params (role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=fa381b3d44664d6ab4beefcf506a841b, user.id=c545c501ee3f4d048d5387a80900dfae) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:11.317483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b571b5-4582-45b3-840d-c69cb519a574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:11.319255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b571b5-4582-45b3-840d-c69cb519a574 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:11.326060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 116/234] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:11 2022] GET /identity/v3/role_assignments?user.id=c545c501ee3f4d048d5387a80900dfae&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:12.391753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f35876-1668-4193-8bfd-762c8e2f6cf7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:12.392264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f35876-1668-4193-8bfd-762c8e2f6cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:12.392264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f35876-1668-4193-8bfd-762c8e2f6cf7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:12.393847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 119/235] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:12.399699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0c9bf-7d31-4429-bc24-cc9cde7e4c76 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:12.400116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0c9bf-7d31-4429-bc24-cc9cde7e4c76 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:12.400305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0c9bf-7d31-4429-bc24-cc9cde7e4c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:12.415696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce010a1-f51d-4238-a7cb-3566099d59ee None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:12.415987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce010a1-f51d-4238-a7cb-3566099d59ee None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:12.416281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce010a1-f51d-4238-a7cb-3566099d59ee None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:12.417620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 120/236] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:12.423215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37833d40-4273-4187-85e1-73ab2aa2512d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:12.423476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37833d40-4273-4187-85e1-73ab2aa2512d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:12.423749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37833d40-4273-4187-85e1-73ab2aa2512d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:12.428625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-eab0c9bf-7d31-4429-bc24-cc9cde7e4c76 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:12.430067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-eab0c9bf-7d31-4429-bc24-cc9cde7e4c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:12.446664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:12.459405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-37833d40-4273-4187-85e1-73ab2aa2512d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:12.460623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-37833d40-4273-4187-85e1-73ab2aa2512d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:12.538800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 117/237] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:12.568133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 121/238] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:12.574934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0c9bf-7d31-4429-bc24-cc9cde7e4c76 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:12.592479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37833d40-4273-4187-85e1-73ab2aa2512d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:12.627756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:12.719371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0c9bf-7d31-4429-bc24-cc9cde7e4c76 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:12.756398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728665b-18c9-4f07-8826-b115b2046eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:12.757044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728665b-18c9-4f07-8826-b115b2046eee None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:12.757327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728665b-18c9-4f07-8826-b115b2046eee None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:12.757600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728665b-18c9-4f07-8826-b115b2046eee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:12.768035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728665b-18c9-4f07-8826-b115b2046eee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-e728665b-18c9-4f07-8826-b115b2046eee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:12.773581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:12.774470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:12.775281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 118/239] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:12.781847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728665b-18c9-4f07-8826-b115b2046eee None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:12.798972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:12.822457 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37833d40-4273-4187-85e1-73ab2aa2512d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:12.888494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191f96e-4747-46fe-a3c3-53993de058ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:12.889354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191f96e-4747-46fe-a3c3-53993de058ef None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:12.890866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191f96e-4747-46fe-a3c3-53993de058ef None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:12.891314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191f96e-4747-46fe-a3c3-53993de058ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:12.893435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191f96e-4747-46fe-a3c3-53993de058ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:12.898902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c191f96e-4747-46fe-a3c3-53993de058ef None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:12.903317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 122/240] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:12.907180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191f96e-4747-46fe-a3c3-53993de058ef None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:12.962511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728665b-18c9-4f07-8826-b115b2046eee None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:13.029841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:13.031604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.032205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.033786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.034622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:13.036181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:13.038261 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:13.056158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 119/241] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:13.061299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:13.092456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:13.094902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191f96e-4747-46fe-a3c3-53993de058ef None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:13.166102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fda12-1e44-4aa4-b0ad-ac448d0cfe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:13.166716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fda12-1e44-4aa4-b0ad-ac448d0cfe5e None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.167144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fda12-1e44-4aa4-b0ad-ac448d0cfe5e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.167412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fda12-1e44-4aa4-b0ad-ac448d0cfe5e None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.168505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fda12-1e44-4aa4-b0ad-ac448d0cfe5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:13.170067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fda12-1e44-4aa4-b0ad-ac448d0cfe5e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:13.197990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 123/242] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:12 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:13.203158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:13.238711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997724b0-8ebd-4dfa-b27a-142b3d46a278 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:13.276786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac610e-7dbd-4ec2-89bb-3329fb4fa24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:13.277422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac610e-7dbd-4ec2-89bb-3329fb4fa24f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.277697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac610e-7dbd-4ec2-89bb-3329fb4fa24f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.277968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac610e-7dbd-4ec2-89bb-3329fb4fa24f None admin] PATH_INFO: `/v3/users/cd66f7221de648dfa935181d23650881` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.294923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac610e-7dbd-4ec2-89bb-3329fb4fa24f None admin] RBAC: Authorizing `identity:get_user(user_id=cd66f7221de648dfa935181d23650881)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:13.296349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac610e-7dbd-4ec2-89bb-3329fb4fa24f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:13.298191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 120/243] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:13 2022] GET /identity/v3/users/cd66f7221de648dfa935181d23650881 => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:13.304712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fda12-1e44-4aa4-b0ad-ac448d0cfe5e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:13.415986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fda12-1e44-4aa4-b0ad-ac448d0cfe5e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:13.454345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d3e2c-78c3-4120-8e78-6bbad612d218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:13.454965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d3e2c-78c3-4120-8e78-6bbad612d218 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.455241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d3e2c-78c3-4120-8e78-6bbad612d218 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.455564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d3e2c-78c3-4120-8e78-6bbad612d218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.476650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:13.528032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d3e2c-78c3-4120-8e78-6bbad612d218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:13.529649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d3e2c-78c3-4120-8e78-6bbad612d218 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:13.633403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 124/244] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:13.639135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac610e-7dbd-4ec2-89bb-3329fb4fa24f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:13.653038 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c704-bb6f-495b-9129-f349f2d79fab None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.653352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c704-bb6f-495b-9129-f349f2d79fab None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.653578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c704-bb6f-495b-9129-f349f2d79fab None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.655178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 125/245] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:13.659973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30bcf1-b593-4237-95b7-ff7903c603da None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.660280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30bcf1-b593-4237-95b7-ff7903c603da None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.660556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30bcf1-b593-4237-95b7-ff7903c603da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.683445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3f30bcf1-b593-4237-95b7-ff7903c603da None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:13.684698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3f30bcf1-b593-4237-95b7-ff7903c603da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:13.752130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:13.767756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac610e-7dbd-4ec2-89bb-3329fb4fa24f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:13.773527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 126/246] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:13.782159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c76f1-0f46-4115-b4bf-7fcbf7b7a49f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.782445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c76f1-0f46-4115-b4bf-7fcbf7b7a49f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.782719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c76f1-0f46-4115-b4bf-7fcbf7b7a49f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.784202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 127/247] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:13.790265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece4bc7-101b-4960-8680-6c5cd6262f1b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.790555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece4bc7-101b-4960-8680-6c5cd6262f1b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.790831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece4bc7-101b-4960-8680-6c5cd6262f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.811545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:13.812519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-dece4bc7-101b-4960-8680-6c5cd6262f1b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:13.813910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dece4bc7-101b-4960-8680-6c5cd6262f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:13.816563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad148-b5f4-4b67-aeb2-cfe9765b84b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:13.817340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad148-b5f4-4b67-aeb2-cfe9765b84b4 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.817671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad148-b5f4-4b67-aeb2-cfe9765b84b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.817947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad148-b5f4-4b67-aeb2-cfe9765b84b4 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.819130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad148-b5f4-4b67-aeb2-cfe9765b84b4 None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:13.820877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad148-b5f4-4b67-aeb2-cfe9765b84b4 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:13.822835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 121/248] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:13 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:13.823712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad148-b5f4-4b67-aeb2-cfe9765b84b4 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:13.901233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 128/249] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:13.902464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5012-25a6-40ec-9273-048955fcd813 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.902757 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5012-25a6-40ec-9273-048955fcd813 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.903020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c5012-25a6-40ec-9273-048955fcd813 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:13.904295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 129/250] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:13.905034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c5012-25a6-40ec-9273-048955fcd813 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:13.953339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad148-b5f4-4b67-aeb2-cfe9765b84b4 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:13.988459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39f939-df4d-4a6e-9704-64d402148526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:13.989877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f939-df4d-4a6e-9704-64d402148526 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:13.990161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f939-df4d-4a6e-9704-64d402148526 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:13.990432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39f939-df4d-4a6e-9704-64d402148526 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.001507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39f939-df4d-4a6e-9704-64d402148526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-9f39f939-df4d-4a6e-9704-64d402148526 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:14.007256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:14.008229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:14.009043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 122/251] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:14.010197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32169d-ee04-4b57-b03b-3ca7b386a610 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.010355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32169d-ee04-4b57-b03b-3ca7b386a610 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.017482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32169d-ee04-4b57-b03b-3ca7b386a610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.022526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c5012-25a6-40ec-9273-048955fcd813 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.034716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:14.044334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ea32169d-ee04-4b57-b03b-3ca7b386a610 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:14.046261 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ea32169d-ee04-4b57-b03b-3ca7b386a610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:14.063728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:14.064489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.064768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.065132 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.065932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] RBAC: Adding query filter params (role.id=a880da5b6bd94a89b2b2658b51f69ee9, scope.project.id=fa381b3d44664d6ab4beefcf506a841b, user.id=cd66f7221de648dfa935181d23650881) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:14.066303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:14.068668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:14.074844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 130/252] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:13 2022] GET /identity/v3/role_assignments?user.id=cd66f7221de648dfa935181d23650881&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=a880da5b6bd94a89b2b2658b51f69ee9 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:14.075718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.127144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:14.153074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 123/253] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:14.154004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32169d-ee04-4b57-b03b-3ca7b386a610 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.198981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2bc7a-bb33-48f2-8f2a-56c1e1bd4bbf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.258690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe090715-9df7-461c-8cbf-462884167d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:14.259592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090715-9df7-461c-8cbf-462884167d48 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.260011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090715-9df7-461c-8cbf-462884167d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.260469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090715-9df7-461c-8cbf-462884167d48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.279153 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe090715-9df7-461c-8cbf-462884167d48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-fe090715-9df7-461c-8cbf-462884167d48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:14.285375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:14.286601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:14.287503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:14.290589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 131/254] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:14.291711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe090715-9df7-461c-8cbf-462884167d48 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.336779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:14.360376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32169d-ee04-4b57-b03b-3ca7b386a610 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.413866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe090715-9df7-461c-8cbf-462884167d48 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.429868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:14.430490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.430771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.431045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.431719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:14.432025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:14.433430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:14.452758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 124/255] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:14.453610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.478764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3980f-cb66-409f-b02c-d712c7f7e37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:14.479405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3980f-cb66-409f-b02c-d712c7f7e37c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.479662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3980f-cb66-409f-b02c-d712c7f7e37c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.479938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3980f-cb66-409f-b02c-d712c7f7e37c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.491631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3980f-cb66-409f-b02c-d712c7f7e37c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-70b3980f-cb66-409f-b02c-d712c7f7e37c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:14.496477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:14.497391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:14.498188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:14.503733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 132/256] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:14.505129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3980f-cb66-409f-b02c-d712c7f7e37c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.543609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:14.596291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e239-ecf6-43d7-b5f4-d547744b51c0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.633411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:14.634025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.634305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.634573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.635244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:14.635546 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:14.636947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:14.645768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 125/257] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:14.646671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.677564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:14.691454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3980f-cb66-409f-b02c-d712c7f7e37c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.748915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e882a2a-a684-4c75-9170-92f77accd7bb None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.755566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bc8fb-08fc-4028-98fd-f142cd997c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:14.756598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc8fb-08fc-4028-98fd-f142cd997c6c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.757083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc8fb-08fc-4028-98fd-f142cd997c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.757577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc8fb-08fc-4028-98fd-f142cd997c6c None admin] PATH_INFO: `/v3/users/aa0dc6ff504e459b84e130def4df94a6` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.759800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bc8fb-08fc-4028-98fd-f142cd997c6c None admin] RBAC: Authorizing `identity:get_user(user_id=aa0dc6ff504e459b84e130def4df94a6)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:14.761563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bc8fb-08fc-4028-98fd-f142cd997c6c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:14.762121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:14.768504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 133/258] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/users/aa0dc6ff504e459b84e130def4df94a6 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:14.769608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bc8fb-08fc-4028-98fd-f142cd997c6c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.795108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:14.795765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.796049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.796341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.798294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] RBAC: Adding query filter params (name=member) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:14.798689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:14.800305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:14.818108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 126/259] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:14.819235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.916362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bc8fb-08fc-4028-98fd-f142cd997c6c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.955409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fa38d-07f3-46f7-8296-eecd0ddcc028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:14.956439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa38d-07f3-46f7-8296-eecd0ddcc028 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:14.956949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa38d-07f3-46f7-8296-eecd0ddcc028 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:14.957458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa38d-07f3-46f7-8296-eecd0ddcc028 None admin] PATH_INFO: `/v3/users/aa0dc6ff504e459b84e130def4df94a6` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:14.958833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fa38d-07f3-46f7-8296-eecd0ddcc028 None admin] RBAC: Authorizing `identity:get_user(user_id=aa0dc6ff504e459b84e130def4df94a6)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:14.960514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fa38d-07f3-46f7-8296-eecd0ddcc028 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:14.962321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 134/260] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/users/aa0dc6ff504e459b84e130def4df94a6 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:14.963363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fa38d-07f3-46f7-8296-eecd0ddcc028 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:14.972977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b5853-08ee-42c4-a451-a006cf22d213 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:14.996073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:15.052407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c562dd-eb81-4c87-aa3a-7afe4d1f794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:15.052947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c562dd-eb81-4c87-aa3a-7afe4d1f794f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:15.053224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c562dd-eb81-4c87-aa3a-7afe4d1f794f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:15.053491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c562dd-eb81-4c87-aa3a-7afe4d1f794f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:15.078416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fa38d-07f3-46f7-8296-eecd0ddcc028 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:15.116311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ae7c1-389a-448b-b453-016be0040c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:15.117385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae7c1-389a-448b-b453-016be0040c62 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:15.117873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae7c1-389a-448b-b453-016be0040c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:15.118370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae7c1-389a-448b-b453-016be0040c62 None admin] PATH_INFO: `/v3/users/c545c501ee3f4d048d5387a80900dfae` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:15.119856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ae7c1-389a-448b-b453-016be0040c62 None admin] RBAC: Authorizing `identity:get_user(user_id=c545c501ee3f4d048d5387a80900dfae)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:15.121605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ae7c1-389a-448b-b453-016be0040c62 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:15.123672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 135/261] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/users/c545c501ee3f4d048d5387a80900dfae => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:15.124802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ae7c1-389a-448b-b453-016be0040c62 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:15.148449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:15.208586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c562dd-eb81-4c87-aa3a-7afe4d1f794f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:15.210206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c562dd-eb81-4c87-aa3a-7afe4d1f794f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:15.238849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:15.244329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ae7c1-389a-448b-b453-016be0040c62 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:15.280311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcfec8-2ddc-4a77-b8df-29e5618e4b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:15.281069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcfec8-2ddc-4a77-b8df-29e5618e4b31 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:15.281452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcfec8-2ddc-4a77-b8df-29e5618e4b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:15.281829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcfec8-2ddc-4a77-b8df-29e5618e4b31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:15.353145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcfec8-2ddc-4a77-b8df-29e5618e4b31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:15.354792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcfec8-2ddc-4a77-b8df-29e5618e4b31 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:15.360130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 127/262] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:15.365077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c562dd-eb81-4c87-aa3a-7afe4d1f794f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:15.460552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:15.472459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:15.477689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 136/263] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:15.478916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcfec8-2ddc-4a77-b8df-29e5618e4b31 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:15.517742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c562dd-eb81-4c87-aa3a-7afe4d1f794f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:15.583059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5a50e-540e-435e-b92a-b6783fd3d5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:15.583805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a50e-540e-435e-b92a-b6783fd3d5aa None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:15.584082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a50e-540e-435e-b92a-b6783fd3d5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:15.584373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5a50e-540e-435e-b92a-b6783fd3d5aa None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:15.588579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5a50e-540e-435e-b92a-b6783fd3d5aa None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:15.590186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5a50e-540e-435e-b92a-b6783fd3d5aa None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:15.591979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 128/264] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:15 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:15.592775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5a50e-540e-435e-b92a-b6783fd3d5aa None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:15.624957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcfec8-2ddc-4a77-b8df-29e5618e4b31 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:15.662630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb643b-99e8-409e-ba99-3d0a24c5b4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:15.663273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb643b-99e8-409e-ba99-3d0a24c5b4ee None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:15.663554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb643b-99e8-409e-ba99-3d0a24c5b4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:15.663826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb643b-99e8-409e-ba99-3d0a24c5b4ee None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:15.664982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb643b-99e8-409e-ba99-3d0a24c5b4ee None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:15.666427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb643b-99e8-409e-ba99-3d0a24c5b4ee None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:15.668395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 137/265] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:15 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:15.669151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb643b-99e8-409e-ba99-3d0a24c5b4ee None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:15.738281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:15.776973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb643b-99e8-409e-ba99-3d0a24c5b4ee None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:15.814729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc1c11-f29d-49c2-9b8e-dd2869ad6925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:15.815606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc1c11-f29d-49c2-9b8e-dd2869ad6925 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:15.815989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc1c11-f29d-49c2-9b8e-dd2869ad6925 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:15.816414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc1c11-f29d-49c2-9b8e-dd2869ad6925 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b/users/c545c501ee3f4d048d5387a80900dfae/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:15.819395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc1c11-f29d-49c2-9b8e-dd2869ad6925 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa381b3d44664d6ab4beefcf506a841b, user_id=c545c501ee3f4d048d5387a80900dfae, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:15.819716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5a50e-540e-435e-b92a-b6783fd3d5aa None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:15.821091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc1c11-f29d-49c2-9b8e-dd2869ad6925 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:15.835997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 138/266] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:15 2022] PUT /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b/users/c545c501ee3f4d048d5387a80900dfae/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:15.836839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc1c11-f29d-49c2-9b8e-dd2869ad6925 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:15.889010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417d2b-e280-4bd9-8423-8caf8824f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:15.889716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417d2b-e280-4bd9-8423-8caf8824f43b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:15.890003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417d2b-e280-4bd9-8423-8caf8824f43b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:15.890319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26417d2b-e280-4bd9-8423-8caf8824f43b None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:15.892388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26417d2b-e280-4bd9-8423-8caf8824f43b None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:15.893994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26417d2b-e280-4bd9-8423-8caf8824f43b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:15.896850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 129/267] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:15 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:15.903063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417d2b-e280-4bd9-8423-8caf8824f43b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:15.940487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:15.953373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc1c11-f29d-49c2-9b8e-dd2869ad6925 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:15.983370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:15.996012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da45c8-6643-4841-af60-944713b1cf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:15.996580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da45c8-6643-4841-af60-944713b1cf7b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:15.996864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da45c8-6643-4841-af60-944713b1cf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:15.997140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da45c8-6643-4841-af60-944713b1cf7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:15.997890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da45c8-6643-4841-af60-944713b1cf7b None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=fa381b3d44664d6ab4beefcf506a841b, user.id=aa0dc6ff504e459b84e130def4df94a6) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:15.998201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da45c8-6643-4841-af60-944713b1cf7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:15.999655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da45c8-6643-4841-af60-944713b1cf7b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:16.007524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 139/268] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:15 2022] GET /identity/v3/role_assignments?user.id=aa0dc6ff504e459b84e130def4df94a6&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=29fd031edd2b4f65aed8942853150f92 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:16.093228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26417d2b-e280-4bd9-8423-8caf8824f43b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:16.106178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4193-6d38-4a8a-9e7d-656c2a9d4564 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.106491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4193-6d38-4a8a-9e7d-656c2a9d4564 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.106733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb4193-6d38-4a8a-9e7d-656c2a9d4564 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.108234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 140/269] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:16.114757 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718958a8-f2f0-40f6-9bfe-83a57606a097 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.115419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718958a8-f2f0-40f6-9bfe-83a57606a097 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.115882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718958a8-f2f0-40f6-9bfe-83a57606a097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.151571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-718958a8-f2f0-40f6-9bfe-83a57606a097 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:16.153657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2390d7-12f0-4b67-b67e-b444657a8824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:16.154108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-718958a8-f2f0-40f6-9bfe-83a57606a097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:16.154290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2390d7-12f0-4b67-b67e-b444657a8824 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.154570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2390d7-12f0-4b67-b67e-b444657a8824 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.154844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2390d7-12f0-4b67-b67e-b444657a8824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.155590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2390d7-12f0-4b67-b67e-b444657a8824 None admin] RBAC: Adding query filter params (role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=fa381b3d44664d6ab4beefcf506a841b, user.id=aa0dc6ff504e459b84e130def4df94a6) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:16.155896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2390d7-12f0-4b67-b67e-b444657a8824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:16.157408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2390d7-12f0-4b67-b67e-b444657a8824 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:16.165115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 130/270] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:15 2022] GET /identity/v3/role_assignments?user.id=aa0dc6ff504e459b84e130def4df94a6&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:16.166493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a0596-8750-4f6e-b7a7-ac43269f7689 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.166786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a0596-8750-4f6e-b7a7-ac43269f7689 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.167068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a0596-8750-4f6e-b7a7-ac43269f7689 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.168554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 131/271] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:16.174544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6873f-d503-482c-ba07-0958f32d8d97 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.174866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6873f-d503-482c-ba07-0958f32d8d97 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.175103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a6873f-d503-482c-ba07-0958f32d8d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.195712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-06a6873f-d503-482c-ba07-0958f32d8d97 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:16.196976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-06a6873f-d503-482c-ba07-0958f32d8d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:16.274081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 132/272] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:16.288293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 141/273] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:16.306901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6873f-d503-482c-ba07-0958f32d8d97 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:16.310141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:16.323703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718958a8-f2f0-40f6-9bfe-83a57606a097 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:16.486842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a6873f-d503-482c-ba07-0958f32d8d97 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:16.502920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718958a8-f2f0-40f6-9bfe-83a57606a097 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:16.526907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dea29-cba4-4fb3-895b-b7395b0b8fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:16.527467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dea29-cba4-4fb3-895b-b7395b0b8fff None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.527744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dea29-cba4-4fb3-895b-b7395b0b8fff None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.528014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dea29-cba4-4fb3-895b-b7395b0b8fff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.529017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:16.529190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dea29-cba4-4fb3-895b-b7395b0b8fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:16.530623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dea29-cba4-4fb3-895b-b7395b0b8fff None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:16.533413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 133/274] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:16.541870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dea29-cba4-4fb3-895b-b7395b0b8fff None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:16.566365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c8b9a-9347-4f38-8cd7-0322f36f5bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:16.567293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c8b9a-9347-4f38-8cd7-0322f36f5bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.567741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c8b9a-9347-4f38-8cd7-0322f36f5bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.568176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c8b9a-9347-4f38-8cd7-0322f36f5bc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.586328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c8b9a-9347-4f38-8cd7-0322f36f5bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-263c8b9a-9347-4f38-8cd7-0322f36f5bc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:16.591731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:16.592677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:16.593479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:16.596633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 142/275] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:16.613278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c8b9a-9347-4f38-8cd7-0322f36f5bc7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:16.646814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:16.687124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dea29-cba4-4fb3-895b-b7395b0b8fff None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:16.737906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f99e84-9e7a-4130-b156-6e92f95a81ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:16.738939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99e84-9e7a-4130-b156-6e92f95a81ad None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.739478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99e84-9e7a-4130-b156-6e92f95a81ad None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.739753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99e84-9e7a-4130-b156-6e92f95a81ad None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.740438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f99e84-9e7a-4130-b156-6e92f95a81ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:16.741430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c8b9a-9347-4f38-8cd7-0322f36f5bc7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:16.741942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f99e84-9e7a-4130-b156-6e92f95a81ad None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:16.770857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 134/276] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:16 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:16.788188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:16.788791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.789071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.789396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.790101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:16.790454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:16.791975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:16.804086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 143/277] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:16.810066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f99e84-9e7a-4130-b156-6e92f95a81ad None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:16.931734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f99e84-9e7a-4130-b156-6e92f95a81ad None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:16.960811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:16.970347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00296156-6582-4425-a560-4c52b3629267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:16.971436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00296156-6582-4425-a560-4c52b3629267 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:16.971709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00296156-6582-4425-a560-4c52b3629267 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:16.971968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00296156-6582-4425-a560-4c52b3629267 None admin] PATH_INFO: `/v3/users/be403f01cd6c45699b315f10b998569f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:16.986123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00296156-6582-4425-a560-4c52b3629267 None admin] RBAC: Authorizing `identity:get_user(user_id=be403f01cd6c45699b315f10b998569f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:16.987587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00296156-6582-4425-a560-4c52b3629267 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:16.990151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 135/278] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:16 2022] GET /identity/v3/users/be403f01cd6c45699b315f10b998569f => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:16.996045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:17.048078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:17.058652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba712b-9433-414b-975e-17c56a36adb2 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:17.058868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba712b-9433-414b-975e-17c56a36adb2 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:17.059137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba712b-9433-414b-975e-17c56a36adb2 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:17.060798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 136/279] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:17.066014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fee9-7e78-4a9b-81cb-a661db3c6565 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:17.066299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fee9-7e78-4a9b-81cb-a661db3c6565 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:17.066578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75fee9-7e78-4a9b-81cb-a661db3c6565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:17.093524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-df75fee9-7e78-4a9b-81cb-a661db3c6565 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:17.094716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-df75fee9-7e78-4a9b-81cb-a661db3c6565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:17.123124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adff4bb-f123-4931-8884-bc31c1166b87 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:17.167675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:17.168352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:17.168637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:17.169200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:17.169931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] RBAC: Adding query filter params (role.id=6fb07818f6ff488d9b09556ed4f1d120, user.id=be403f01cd6c45699b315f10b998569f, scope.system=all) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:17.170399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:17.172224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:17.179844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 144/280] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Aug 11 16:15:16 2022] GET /identity/v3/role_assignments?user.id=be403f01cd6c45699b315f10b998569f&scope.system=all&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:17.185201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 137/281] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:17.208522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:17.323826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b446f-6a77-482b-a35a-76349e964e67 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:17.332675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:17.363343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c4bbd-c194-4f38-8184-c47bf28d2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:17.364456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c4bbd-c194-4f38-8184-c47bf28d2db0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:17.364967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c4bbd-c194-4f38-8184-c47bf28d2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:17.365430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c4bbd-c194-4f38-8184-c47bf28d2db0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:17.377597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c4bbd-c194-4f38-8184-c47bf28d2db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e88c4bbd-c194-4f38-8184-c47bf28d2db0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:17.383407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:17.384325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:17.385137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:17.387830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 145/282] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:17.394083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75fee9-7e78-4a9b-81cb-a661db3c6565 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:17.535583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75fee9-7e78-4a9b-81cb-a661db3c6565 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:17.561384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:17.587809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:17.588362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:17.588639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:17.588915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:17.589577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:17.589874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:17.591231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:17.603597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 138/283] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:17.609545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c4bbd-c194-4f38-8184-c47bf28d2db0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:17.757877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c4bbd-c194-4f38-8184-c47bf28d2db0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:17.808782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf342-a8b2-400b-87ce-587fe60bd801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:17.809784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf342-a8b2-400b-87ce-587fe60bd801 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:17.810285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf342-a8b2-400b-87ce-587fe60bd801 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:17.810749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf342-a8b2-400b-87ce-587fe60bd801 None admin] PATH_INFO: `/v3/users/cd66f7221de648dfa935181d23650881` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:17.812151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf342-a8b2-400b-87ce-587fe60bd801 None admin] RBAC: Authorizing `identity:get_user(user_id=cd66f7221de648dfa935181d23650881)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:17.814044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf342-a8b2-400b-87ce-587fe60bd801 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:17.815895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 146/284] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:17 2022] GET /identity/v3/users/cd66f7221de648dfa935181d23650881 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:17.822410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:17.940461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e4f28-b9f2-4219-958f-1294a1e986fc None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:17.994153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc93224-06f5-479e-95af-d876732782fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:17.994841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc93224-06f5-479e-95af-d876732782fb None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:17.995124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc93224-06f5-479e-95af-d876732782fb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:17.995398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc93224-06f5-479e-95af-d876732782fb None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:17.996598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc93224-06f5-479e-95af-d876732782fb None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:17.998045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc93224-06f5-479e-95af-d876732782fb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:17.999999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 139/285] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:17 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:18.006185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf342-a8b2-400b-87ce-587fe60bd801 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:18.065579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:18.144413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf342-a8b2-400b-87ce-587fe60bd801 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:18.178776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e953ad-8da2-4087-b07d-535aa7430e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:18.179437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e953ad-8da2-4087-b07d-535aa7430e7a None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:18.179710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e953ad-8da2-4087-b07d-535aa7430e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:18.179974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e953ad-8da2-4087-b07d-535aa7430e7a None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b/users/cd66f7221de648dfa935181d23650881/roles/a880da5b6bd94a89b2b2658b51f69ee9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:18.181934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e953ad-8da2-4087-b07d-535aa7430e7a None admin] RBAC: Authorizing `identity:create_grant(project_id=fa381b3d44664d6ab4beefcf506a841b, user_id=cd66f7221de648dfa935181d23650881, role_id=a880da5b6bd94a89b2b2658b51f69ee9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:18.183380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e953ad-8da2-4087-b07d-535aa7430e7a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:18.199073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 147/286] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:18 2022] PUT /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b/users/cd66f7221de648dfa935181d23650881/roles/a880da5b6bd94a89b2b2658b51f69ee9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:18.511034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3323a3-8dad-4230-a3ef-4d1416e88939 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:18.511547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3323a3-8dad-4230-a3ef-4d1416e88939 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:18.511965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3323a3-8dad-4230-a3ef-4d1416e88939 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:18.513487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 140/287] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:18.519526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11c185-d2ee-405f-8bc9-464e5667f38d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:18.520421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11c185-d2ee-405f-8bc9-464e5667f38d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:18.520803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11c185-d2ee-405f-8bc9-464e5667f38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:18.544171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5c11c185-d2ee-405f-8bc9-464e5667f38d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:18.545381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5c11c185-d2ee-405f-8bc9-464e5667f38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:18.635791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 148/288] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:18.676120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3323a3-8dad-4230-a3ef-4d1416e88939 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:18.735342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:18.799402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3323a3-8dad-4230-a3ef-4d1416e88939 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:18.834830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b83a1-4422-4607-bbcd-9bd2aa0f10e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:18.835458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b83a1-4422-4607-bbcd-9bd2aa0f10e0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:18.835739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b83a1-4422-4607-bbcd-9bd2aa0f10e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:18.836009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b83a1-4422-4607-bbcd-9bd2aa0f10e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:18.846542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b83a1-4422-4607-bbcd-9bd2aa0f10e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-3f4b83a1-4422-4607-bbcd-9bd2aa0f10e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:18.852050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:18.852965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:18.854589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 141/289] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:18.859350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11c185-d2ee-405f-8bc9-464e5667f38d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:18.906074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:19.010324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11c185-d2ee-405f-8bc9-464e5667f38d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:19.048640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:19.049251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.049580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.049799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.050538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:19.050831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:19.053850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:19.065308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 149/290] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:19.073224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b83a1-4422-4607-bbcd-9bd2aa0f10e0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:19.209007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b83a1-4422-4607-bbcd-9bd2aa0f10e0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:19.216556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:19.246056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd3b0c-dd08-4aea-b7ff-1487d31c84e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:19.246692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd3b0c-dd08-4aea-b7ff-1487d31c84e4 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.246971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd3b0c-dd08-4aea-b7ff-1487d31c84e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.247239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd3b0c-dd08-4aea-b7ff-1487d31c84e4 None admin] PATH_INFO: `/v3/users/c545c501ee3f4d048d5387a80900dfae` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.248439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd3b0c-dd08-4aea-b7ff-1487d31c84e4 None admin] RBAC: Authorizing `identity:get_user(user_id=c545c501ee3f4d048d5387a80900dfae)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:19.249886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd3b0c-dd08-4aea-b7ff-1487d31c84e4 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:19.251793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 142/291] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:19 2022] GET /identity/v3/users/c545c501ee3f4d048d5387a80900dfae => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:19.257413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:19.400516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:19.409394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f357e54-a996-4236-b674-fc2403c0461c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:19.455668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0a646-faba-47e7-9b7d-45eed3a2ef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:19.456301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a646-faba-47e7-9b7d-45eed3a2ef46 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.456581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a646-faba-47e7-9b7d-45eed3a2ef46 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.456851 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0a646-faba-47e7-9b7d-45eed3a2ef46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.533628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccd2ce-1d76-4992-94c9-a165181b5843 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.533950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccd2ce-1d76-4992-94c9-a165181b5843 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.534177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccd2ce-1d76-4992-94c9-a165181b5843 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.535673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 143/292] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:19.540904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe6411-ccee-431d-a551-04498fa79535 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.541194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe6411-ccee-431d-a551-04498fa79535 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.541471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe6411-ccee-431d-a551-04498fa79535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.551604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0a646-faba-47e7-9b7d-45eed3a2ef46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:19.553109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0a646-faba-47e7-9b7d-45eed3a2ef46 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:19.560011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-53fe6411-ccee-431d-a551-04498fa79535 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:19.561245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-53fe6411-ccee-431d-a551-04498fa79535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:19.633599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 144/293] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:19.663970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:19.666932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe6411-ccee-431d-a551-04498fa79535 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:19.699753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 150/294] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:19.705550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0a646-faba-47e7-9b7d-45eed3a2ef46 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:19.775154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe6411-ccee-431d-a551-04498fa79535 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:19.777589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:19.813085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa43174-8a2c-485b-9cb8-e481f0faca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:19.814263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa43174-8a2c-485b-9cb8-e481f0faca7a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.814545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa43174-8a2c-485b-9cb8-e481f0faca7a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.814815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa43174-8a2c-485b-9cb8-e481f0faca7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.816436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa43174-8a2c-485b-9cb8-e481f0faca7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:19.818834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa43174-8a2c-485b-9cb8-e481f0faca7a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:19.828125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 145/295] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:19.828125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65049e93-8b7e-4bfd-a89d-4ca6ca8a35a0 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.828125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65049e93-8b7e-4bfd-a89d-4ca6ca8a35a0 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.828125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65049e93-8b7e-4bfd-a89d-4ca6ca8a35a0 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.828125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 146/296] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:19.828550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0a646-faba-47e7-9b7d-45eed3a2ef46 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:19.828630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1a660-a55e-4908-b6a5-87212a7740c2 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.828630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1a660-a55e-4908-b6a5-87212a7740c2 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.828630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1a660-a55e-4908-b6a5-87212a7740c2 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.828630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 147/297] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:19.829299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1a660-a55e-4908-b6a5-87212a7740c2 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:19.867274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73353009-2106-44bf-853c-238da836b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:19.867906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73353009-2106-44bf-853c-238da836b0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.868216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73353009-2106-44bf-853c-238da836b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.868491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73353009-2106-44bf-853c-238da836b0b8 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.869614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73353009-2106-44bf-853c-238da836b0b8 None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:19.871051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73353009-2106-44bf-853c-238da836b0b8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:19.872891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 151/298] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:19 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:19.873840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbda6-abaf-4772-a417-bbb6606573eb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.874122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbda6-abaf-4772-a417-bbb6606573eb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.874392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cbda6-abaf-4772-a417-bbb6606573eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.894134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-884cbda6-abaf-4772-a417-bbb6606573eb None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:19.895351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-884cbda6-abaf-4772-a417-bbb6606573eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:19.943410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1a660-a55e-4908-b6a5-87212a7740c2 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:19.952228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:19.973134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 152/299] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:19.974458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da56f-f466-4dda-b472-7ee325cfe4de None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.974740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da56f-f466-4dda-b472-7ee325cfe4de None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.975008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75da56f-f466-4dda-b472-7ee325cfe4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.991082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f49991-3d41-4c1b-b162-b5f8fd1a33d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:19.992140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f49991-3d41-4c1b-b162-b5f8fd1a33d0 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:19.992597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f49991-3d41-4c1b-b162-b5f8fd1a33d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:19.993042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f49991-3d41-4c1b-b162-b5f8fd1a33d0 None admin] PATH_INFO: `/v3/groups` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:19.993898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f49991-3d41-4c1b-b162-b5f8fd1a33d0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:19.994468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a75da56f-f466-4dda-b472-7ee325cfe4de None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:19.995643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a75da56f-f466-4dda-b472-7ee325cfe4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:19.996018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f49991-3d41-4c1b-b162-b5f8fd1a33d0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:20.013782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 148/300] 10.222.0.43 () {60 vars in 1134 bytes} [Thu Aug 11 16:15:19 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:20.014920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f49991-3d41-4c1b-b162-b5f8fd1a33d0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.068439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:20.092547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 153/301] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:20.093805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da56f-f466-4dda-b472-7ee325cfe4de None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.161095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f49991-3d41-4c1b-b162-b5f8fd1a33d0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:20.204046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:20.204046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.204046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.204392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.204942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] RBAC: Adding query filter params (role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=fa381b3d44664d6ab4beefcf506a841b, user.id=c545c501ee3f4d048d5387a80900dfae) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:20.205247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:20.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:20.212944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 149/302] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:20 2022] GET /identity/v3/role_assignments?user.id=c545c501ee3f4d048d5387a80900dfae&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:20.214146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.247175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75da56f-f466-4dda-b472-7ee325cfe4de None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:20.267368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:20.305247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0e768-be7d-4115-a5d4-4a253243c664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:20.305903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0e768-be7d-4115-a5d4-4a253243c664 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.306183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0e768-be7d-4115-a5d4-4a253243c664 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.306463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0e768-be7d-4115-a5d4-4a253243c664 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.323256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0e768-be7d-4115-a5d4-4a253243c664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a8e0e768-be7d-4115-a5d4-4a253243c664 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:20.330498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:20.331458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:20.332477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:20.333493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 154/303] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:20.340052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0e768-be7d-4115-a5d4-4a253243c664 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.345583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:20.355153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24422abc-5316-4b90-b2ec-0415a923de15 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:20.409727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742494d-2cff-4bad-afc7-368cda8c87a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:20.411796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742494d-2cff-4bad-afc7-368cda8c87a6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.412326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742494d-2cff-4bad-afc7-368cda8c87a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.413053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742494d-2cff-4bad-afc7-368cda8c87a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.431663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742494d-2cff-4bad-afc7-368cda8c87a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-4742494d-2cff-4bad-afc7-368cda8c87a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:20.439150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:20.440083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:20.440945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:20.445524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 150/304] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:20.451471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742494d-2cff-4bad-afc7-368cda8c87a6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.477714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0e768-be7d-4115-a5d4-4a253243c664 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:20.522178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:20.522938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.523120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.523391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.524067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:20.524417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:20.525798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:20.535214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 155/305] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:20.540775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.585716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:20.597583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742494d-2cff-4bad-afc7-368cda8c87a6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:20.646198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:20.647536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.648027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.648528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.649428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:20.650438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:20.652373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:20.664783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 151/306] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:20.665665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21088208-4310-45be-8d40-27a3870a4cd6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:20.670790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.699013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:20.711651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa8aac-137a-40d7-8e3f-728a4eea8739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:20.712908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa8aac-137a-40d7-8e3f-728a4eea8739 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.713618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa8aac-137a-40d7-8e3f-728a4eea8739 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.714069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa8aac-137a-40d7-8e3f-728a4eea8739 None admin] PATH_INFO: `/v3/users/be403f01cd6c45699b315f10b998569f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.715439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa8aac-137a-40d7-8e3f-728a4eea8739 None admin] RBAC: Authorizing `identity:get_user(user_id=be403f01cd6c45699b315f10b998569f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:20.717487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa8aac-137a-40d7-8e3f-728a4eea8739 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:20.719258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 156/307] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:20 2022] GET /identity/v3/users/be403f01cd6c45699b315f10b998569f => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:20.726367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa8aac-137a-40d7-8e3f-728a4eea8739 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.805371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681499e3-35aa-4808-9aa3-8c8e2194b86d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:20.849827 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa8aac-137a-40d7-8e3f-728a4eea8739 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:20.850708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b60642-f48a-462e-89ee-b1d856491fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:20.851343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b60642-f48a-462e-89ee-b1d856491fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.851624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b60642-f48a-462e-89ee-b1d856491fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.851896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b60642-f48a-462e-89ee-b1d856491fc6 None admin] PATH_INFO: `/v3/users/b4036303cbbe4d59a938570ea023bf2e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.853746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:20.869580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b60642-f48a-462e-89ee-b1d856491fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=b4036303cbbe4d59a938570ea023bf2e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:20.871333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b60642-f48a-462e-89ee-b1d856491fc6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:20.878103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 152/308] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:20 2022] GET /identity/v3/users/b4036303cbbe4d59a938570ea023bf2e => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:20.879686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b60642-f48a-462e-89ee-b1d856491fc6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:20.897957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9effbae-c0c4-4d75-b474-0b1b7f76fbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:20.898623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9effbae-c0c4-4d75-b474-0b1b7f76fbcc None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.898919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9effbae-c0c4-4d75-b474-0b1b7f76fbcc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.899469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9effbae-c0c4-4d75-b474-0b1b7f76fbcc None admin] PATH_INFO: `/v3/system/users/be403f01cd6c45699b315f10b998569f/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.901017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9effbae-c0c4-4d75-b474-0b1b7f76fbcc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=be403f01cd6c45699b315f10b998569f, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:20.902998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9effbae-c0c4-4d75-b474-0b1b7f76fbcc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:20.916847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 157/309] 10.222.0.43 () {58 vars in 1334 bytes} [Thu Aug 11 16:15:20 2022] PUT /identity/v3/system/users/be403f01cd6c45699b315f10b998569f/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:15:20.936349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5c3d-04d6-43c4-8879-16c9843ead20 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.936707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5c3d-04d6-43c4-8879-16c9843ead20 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.936979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5c3d-04d6-43c4-8879-16c9843ead20 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.938988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 158/310] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:20.945672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14a52-3bf4-47b1-83fc-7a7797dbdaf7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:20.945988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14a52-3bf4-47b1-83fc-7a7797dbdaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:20.946300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b14a52-3bf4-47b1-83fc-7a7797dbdaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:20.966354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-91b14a52-3bf4-47b1-83fc-7a7797dbdaf7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:20.967563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-91b14a52-3bf4-47b1-83fc-7a7797dbdaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:20.993658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:21.026549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b60642-f48a-462e-89ee-b1d856491fc6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:21.075574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 159/311] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:21.080680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:21.081370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:21.081719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:21.082080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:21.082921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] RBAC: Adding query filter params (role.id=a880da5b6bd94a89b2b2658b51f69ee9, user.id=b4036303cbbe4d59a938570ea023bf2e, scope.system=all) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:21.083412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:21.085018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:21.090868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 153/312] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Aug 11 16:15:20 2022] GET /identity/v3/role_assignments?user.id=b4036303cbbe4d59a938570ea023bf2e&scope.system=all&role.id=a880da5b6bd94a89b2b2658b51f69ee9 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:21.099433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b14a52-3bf4-47b1-83fc-7a7797dbdaf7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:21.208550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:21.212696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b14a52-3bf4-47b1-83fc-7a7797dbdaf7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:21.250758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db813b84-5fd3-44b9-af5b-5a1287ebeff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:21.251408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db813b84-5fd3-44b9-af5b-5a1287ebeff0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:21.251749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db813b84-5fd3-44b9-af5b-5a1287ebeff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:21.252018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db813b84-5fd3-44b9-af5b-5a1287ebeff0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:21.263974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db813b84-5fd3-44b9-af5b-5a1287ebeff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-db813b84-5fd3-44b9-af5b-5a1287ebeff0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:21.268700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:21.269579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:21.270984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 160/313] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:21.278056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:21.390055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8da9-a5a6-4967-aa50-b78c34cef262 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:21.432130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:21.432130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:21.432130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:21.432130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:21.432500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:21.432500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:21.432500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:21.441433 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 154/314] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:21.447459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db813b84-5fd3-44b9-af5b-5a1287ebeff0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:21.554880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db813b84-5fd3-44b9-af5b-5a1287ebeff0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:21.591614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a276d-4be3-429f-8457-7f6cf2afcc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:21.592643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a276d-4be3-429f-8457-7f6cf2afcc82 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:21.593142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a276d-4be3-429f-8457-7f6cf2afcc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:21.593617 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a276d-4be3-429f-8457-7f6cf2afcc82 None admin] PATH_INFO: `/v3/users/cd66f7221de648dfa935181d23650881` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:21.594958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a276d-4be3-429f-8457-7f6cf2afcc82 None admin] RBAC: Authorizing `identity:get_user(user_id=cd66f7221de648dfa935181d23650881)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:21.596769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a276d-4be3-429f-8457-7f6cf2afcc82 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:21.599301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 161/315] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:21 2022] GET /identity/v3/users/cd66f7221de648dfa935181d23650881 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:21.604997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:21.609191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:21.712820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2537111-1696-4f83-9121-1018a8180f7b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:21.748285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dd76e-fb88-4f8f-93db-4a68c6a9ff86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:21.748904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dd76e-fb88-4f8f-93db-4a68c6a9ff86 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:21.749185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dd76e-fb88-4f8f-93db-4a68c6a9ff86 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:21.749459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dd76e-fb88-4f8f-93db-4a68c6a9ff86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:21.820217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dd76e-fb88-4f8f-93db-4a68c6a9ff86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:21.822132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dd76e-fb88-4f8f-93db-4a68c6a9ff86 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:21.823977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:21.924952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 155/316] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:21.930742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a276d-4be3-429f-8457-7f6cf2afcc82 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:21.961093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:22.037161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a276d-4be3-429f-8457-7f6cf2afcc82 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:22.078396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6051c-1d08-4847-b9d0-7e173225a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:22.079033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6051c-1d08-4847-b9d0-7e173225a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:22.079313 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6051c-1d08-4847-b9d0-7e173225a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:22.079643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6051c-1d08-4847-b9d0-7e173225a1d1 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:22.080843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6051c-1d08-4847-b9d0-7e173225a1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:22.082852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6051c-1d08-4847-b9d0-7e173225a1d1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:22.085139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 162/317] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:21 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:22.090899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dd76e-fb88-4f8f-93db-4a68c6a9ff86 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:22.196700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dd76e-fb88-4f8f-93db-4a68c6a9ff86 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:22.198233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:22.232374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7fd7e-e4ac-4fa1-878c-4970e936cf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:22.232998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7fd7e-e4ac-4fa1-878c-4970e936cf76 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:22.233338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7fd7e-e4ac-4fa1-878c-4970e936cf76 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:22.233610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7fd7e-e4ac-4fa1-878c-4970e936cf76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:22.234414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7fd7e-e4ac-4fa1-878c-4970e936cf76 None admin] RBAC: Adding query filter params (role.id=a880da5b6bd94a89b2b2658b51f69ee9, scope.project.id=fa381b3d44664d6ab4beefcf506a841b, user.id=cd66f7221de648dfa935181d23650881) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:22.234718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7fd7e-e4ac-4fa1-878c-4970e936cf76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:22.236287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7fd7e-e4ac-4fa1-878c-4970e936cf76 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:22.244037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 156/318] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:22 2022] GET /identity/v3/role_assignments?user.id=cd66f7221de648dfa935181d23650881&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=a880da5b6bd94a89b2b2658b51f69ee9 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:22.296350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ea3e1-4cba-4cef-92b1-31450b5786a9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:22.296641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ea3e1-4cba-4cef-92b1-31450b5786a9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:22.296826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ea3e1-4cba-4cef-92b1-31450b5786a9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:22.298685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 163/319] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:22.305159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33da51-9015-4696-b76a-1fadde7b858b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:22.305662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33da51-9015-4696-b76a-1fadde7b858b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:22.305737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33da51-9015-4696-b76a-1fadde7b858b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:22.325471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-8a33da51-9015-4696-b76a-1fadde7b858b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:22.326689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8a33da51-9015-4696-b76a-1fadde7b858b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:22.404254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 157/320] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:22.422531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ea3e1-4cba-4cef-92b1-31450b5786a9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:22.528971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:22.538053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ea3e1-4cba-4cef-92b1-31450b5786a9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:22.574022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054bd63-5184-4a03-8ade-7aa05d05808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:22.574661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054bd63-5184-4a03-8ade-7aa05d05808f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:22.574933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054bd63-5184-4a03-8ade-7aa05d05808f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:22.575208 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054bd63-5184-4a03-8ade-7aa05d05808f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:22.576306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054bd63-5184-4a03-8ade-7aa05d05808f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:22.577739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054bd63-5184-4a03-8ade-7aa05d05808f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:22.584858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 164/321] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:22.586484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33da51-9015-4696-b76a-1fadde7b858b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:22.682848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:22.690451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33da51-9015-4696-b76a-1fadde7b858b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:22.725484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988779a-ffbc-4436-beff-3460e74e647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:22.726095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988779a-ffbc-4436-beff-3460e74e647c None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:22.726370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988779a-ffbc-4436-beff-3460e74e647c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:22.726644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988779a-ffbc-4436-beff-3460e74e647c None admin] PATH_INFO: `/v3/groups` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:22.727274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988779a-ffbc-4436-beff-3460e74e647c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:22.728737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988779a-ffbc-4436-beff-3460e74e647c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:22.744149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 158/322] 10.222.0.43 () {60 vars in 1134 bytes} [Thu Aug 11 16:15:22 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:23.074877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c513e70-a154-42b9-a7f4-813b5e75141f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.075362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c513e70-a154-42b9-a7f4-813b5e75141f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.075459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c513e70-a154-42b9-a7f4-813b5e75141f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.077147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 165/323] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:23.082927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25865ae0-d3c3-46e0-8d07-dbcdd43facb9 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.083297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25865ae0-d3c3-46e0-8d07-dbcdd43facb9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.083573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25865ae0-d3c3-46e0-8d07-dbcdd43facb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.103279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-25865ae0-d3c3-46e0-8d07-dbcdd43facb9 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:23.104533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-25865ae0-d3c3-46e0-8d07-dbcdd43facb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:23.157106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808766f-6a96-46b6-9630-d3ac75f91d22 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.157411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808766f-6a96-46b6-9630-d3ac75f91d22 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.157659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808766f-6a96-46b6-9630-d3ac75f91d22 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.160922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 166/324] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:23.166040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6959bd-75c2-4e68-bfb8-45048fca4932 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.166330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6959bd-75c2-4e68-bfb8-45048fca4932 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.166600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6959bd-75c2-4e68-bfb8-45048fca4932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.186229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 159/325] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:23.187446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1e6959bd-75c2-4e68-bfb8-45048fca4932 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:23.188920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1e6959bd-75c2-4e68-bfb8-45048fca4932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:23.209914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25865ae0-d3c3-46e0-8d07-dbcdd43facb9 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:23.274325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 167/326] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:23.297897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6959bd-75c2-4e68-bfb8-45048fca4932 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:23.326182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:23.331018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25865ae0-d3c3-46e0-8d07-dbcdd43facb9 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:23.370023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84574e93-522c-4e2c-9ea5-b1351d1d2da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:23.370660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84574e93-522c-4e2c-9ea5-b1351d1d2da9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.370936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84574e93-522c-4e2c-9ea5-b1351d1d2da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.371210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84574e93-522c-4e2c-9ea5-b1351d1d2da9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.382133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84574e93-522c-4e2c-9ea5-b1351d1d2da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-84574e93-522c-4e2c-9ea5-b1351d1d2da9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:23.388650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:23.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:23.390379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 160/327] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:23.397116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84574e93-522c-4e2c-9ea5-b1351d1d2da9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:23.437996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:23.449452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6959bd-75c2-4e68-bfb8-45048fca4932 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:23.494207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db575a4b-da46-41c2-a9fc-aa5a9866fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:23.495086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db575a4b-da46-41c2-a9fc-aa5a9866fa2a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.495501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db575a4b-da46-41c2-a9fc-aa5a9866fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.495934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db575a4b-da46-41c2-a9fc-aa5a9866fa2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.509841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db575a4b-da46-41c2-a9fc-aa5a9866fa2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-db575a4b-da46-41c2-a9fc-aa5a9866fa2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:23.515602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:23.516634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:23.517468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:23.518077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 168/328] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:23.524172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db575a4b-da46-41c2-a9fc-aa5a9866fa2a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:23.540312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84574e93-522c-4e2c-9ea5-b1351d1d2da9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:23.585308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:23.585720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.585999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.586268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.586988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] RBAC: Adding query filter params (name=member) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:23.587232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:23.589694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:23.599923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 161/329] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:23.605089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:23.623559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:23.676874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db575a4b-da46-41c2-a9fc-aa5a9866fa2a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:23.753690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:23.754924 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.755408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.755901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.756159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747593e1-1b67-489c-8f6b-af474bb453de None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:23.757001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:23.757501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:23.759115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:23.778580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 169/330] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:23.784217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:23.795434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382d0ad-7c3f-4497-b1bb-e2bba59207da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:23.796071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382d0ad-7c3f-4497-b1bb-e2bba59207da None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.796426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382d0ad-7c3f-4497-b1bb-e2bba59207da None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.796705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382d0ad-7c3f-4497-b1bb-e2bba59207da None admin] PATH_INFO: `/v3/users/be403f01cd6c45699b315f10b998569f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.798854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382d0ad-7c3f-4497-b1bb-e2bba59207da None admin] RBAC: Authorizing `identity:get_user(user_id=be403f01cd6c45699b315f10b998569f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:23.800447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382d0ad-7c3f-4497-b1bb-e2bba59207da None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:23.803554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 162/331] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:23 2022] GET /identity/v3/users/be403f01cd6c45699b315f10b998569f => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:23.810373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382d0ad-7c3f-4497-b1bb-e2bba59207da None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:23.825299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:23.903184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:23.931986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382d0ad-7c3f-4497-b1bb-e2bba59207da None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:23.971789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:23.972701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:23.973661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:23.974311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:23.975535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] RBAC: Adding query filter params (role.id=6fb07818f6ff488d9b09556ed4f1d120, user.id=be403f01cd6c45699b315f10b998569f, scope.system=all) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:23.975991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:23.977728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:23.992526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 163/332] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Aug 11 16:15:23 2022] GET /identity/v3/role_assignments?user.id=be403f01cd6c45699b315f10b998569f&scope.system=all&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:24.033189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efff74-c96f-4d0d-88ea-43f7bf317727 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:24.090154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddbffe-60af-4dd2-b21a-2d91f839cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:24.091501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddbffe-60af-4dd2-b21a-2d91f839cb9f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:24.091953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddbffe-60af-4dd2-b21a-2d91f839cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:24.092420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddbffe-60af-4dd2-b21a-2d91f839cb9f None admin] PATH_INFO: `/v3/users/b4036303cbbe4d59a938570ea023bf2e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:24.093802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddbffe-60af-4dd2-b21a-2d91f839cb9f None admin] RBAC: Authorizing `identity:get_user(user_id=b4036303cbbe4d59a938570ea023bf2e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:24.095392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddbffe-60af-4dd2-b21a-2d91f839cb9f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:24.098185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 170/333] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:23 2022] GET /identity/v3/users/b4036303cbbe4d59a938570ea023bf2e => generated 293 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:24.103812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:24.221855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977ff7-5366-47d7-80f5-060233bbf4e9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:24.261653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437628c0-8cf4-46d5-b27e-262fa5e939ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:24.262585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437628c0-8cf4-46d5-b27e-262fa5e939ea None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:24.263009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437628c0-8cf4-46d5-b27e-262fa5e939ea None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:24.263446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437628c0-8cf4-46d5-b27e-262fa5e939ea None admin] PATH_INFO: `/v3/system/users/b4036303cbbe4d59a938570ea023bf2e/roles/a880da5b6bd94a89b2b2658b51f69ee9` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:24.265116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437628c0-8cf4-46d5-b27e-262fa5e939ea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b4036303cbbe4d59a938570ea023bf2e, role_id=a880da5b6bd94a89b2b2658b51f69ee9)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:24.266634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437628c0-8cf4-46d5-b27e-262fa5e939ea None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:24.268370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:24.280075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 164/334] 10.222.0.43 () {58 vars in 1334 bytes} [Thu Aug 11 16:15:24 2022] PUT /identity/v3/system/users/b4036303cbbe4d59a938570ea023bf2e/roles/a880da5b6bd94a89b2b2658b51f69ee9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:25.158850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af67bb-76c1-444e-9650-135edb92ae45 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:25.159351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af67bb-76c1-444e-9650-135edb92ae45 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:25.159426 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af67bb-76c1-444e-9650-135edb92ae45 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:25.161101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 171/335] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:25.178289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12369b9b-1887-4e5f-80c3-f1fdf9977ca4 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:25.179225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12369b9b-1887-4e5f-80c3-f1fdf9977ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:25.179695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12369b9b-1887-4e5f-80c3-f1fdf9977ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:25.201573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-12369b9b-1887-4e5f-80c3-f1fdf9977ca4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:25.203097 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-12369b9b-1887-4e5f-80c3-f1fdf9977ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:25.290572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 165/336] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:25.330729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af67bb-76c1-444e-9650-135edb92ae45 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:25.437634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:25.466113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af67bb-76c1-444e-9650-135edb92ae45 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:25.511203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca3c03-902b-4d5b-9a70-2cc3b60c9844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:25.511830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3c03-902b-4d5b-9a70-2cc3b60c9844 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:25.512138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3c03-902b-4d5b-9a70-2cc3b60c9844 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:25.512389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca3c03-902b-4d5b-9a70-2cc3b60c9844 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:25.528965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca3c03-902b-4d5b-9a70-2cc3b60c9844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c6ca3c03-902b-4d5b-9a70-2cc3b60c9844 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:25.533710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:25.534604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:25.535942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:25.536583 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 172/337] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:25.541660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12369b9b-1887-4e5f-80c3-f1fdf9977ca4 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:25.685086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12369b9b-1887-4e5f-80c3-f1fdf9977ca4 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:25.722139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:25.733041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:25.733697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:25.733939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:25.734208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:25.734886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] RBAC: Adding query filter params (name=member) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:25.735192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:25.736598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:25.748133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 166/338] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:25.750449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e9ae0-4147-499b-a98e-53128b0d2cd5 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:25.750815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e9ae0-4147-499b-a98e-53128b0d2cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:25.751084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e9ae0-4147-499b-a98e-53128b0d2cd5 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:25.752685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 173/339] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:25.755359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:25.762306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6045f-2a26-4278-aea2-be8916ec76e2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:25.762584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6045f-2a26-4278-aea2-be8916ec76e2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:25.762853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6045f-2a26-4278-aea2-be8916ec76e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:25.790816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d4b6045f-2a26-4278-aea2-be8916ec76e2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:25.792269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d4b6045f-2a26-4278-aea2-be8916ec76e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:25.908922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c1fb3-3312-4e77-bf34-392f6911e56c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:25.921668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 174/340] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:25.950706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79c940-5ba6-4183-b698-bff76ea5c5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:25.951243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c940-5ba6-4183-b698-bff76ea5c5f3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:25.951530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c940-5ba6-4183-b698-bff76ea5c5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:25.951800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c940-5ba6-4183-b698-bff76ea5c5f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:25.966726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6045f-2a26-4278-aea2-be8916ec76e2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:26.088482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:26.110709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79c940-5ba6-4183-b698-bff76ea5c5f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:26.114694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79c940-5ba6-4183-b698-bff76ea5c5f3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:26.140956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:26.185410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6045f-2a26-4278-aea2-be8916ec76e2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:26.233315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39638345-a8f8-48c9-a3af-25193171631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:26.233883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39638345-a8f8-48c9-a3af-25193171631a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:26.234162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39638345-a8f8-48c9-a3af-25193171631a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:26.234433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39638345-a8f8-48c9-a3af-25193171631a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:26.257537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39638345-a8f8-48c9-a3af-25193171631a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-39638345-a8f8-48c9-a3af-25193171631a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:26.268652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:26.270137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:26.271429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 175/341] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:26.278988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39638345-a8f8-48c9-a3af-25193171631a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:26.333661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 167/342] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 16:15:26.339781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79c940-5ba6-4183-b698-bff76ea5c5f3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:26.471771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39638345-a8f8-48c9-a3af-25193171631a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:26.513288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:26.519073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b2b0b-98a9-48a3-941c-29c65c49cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:26.520372 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b2b0b-98a9-48a3-941c-29c65c49cc5e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:26.520636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b2b0b-98a9-48a3-941c-29c65c49cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:26.520902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b2b0b-98a9-48a3-941c-29c65c49cc5e None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:26.521730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b2b0b-98a9-48a3-941c-29c65c49cc5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:26.522041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b2b0b-98a9-48a3-941c-29c65c49cc5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:26.523430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b2b0b-98a9-48a3-941c-29c65c49cc5e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:26.538703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 176/343] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:26.542422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33ec5-5b47-44f1-9a91-f0dc77e741f0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:26.542705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33ec5-5b47-44f1-9a91-f0dc77e741f0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:26.542972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33ec5-5b47-44f1-9a91-f0dc77e741f0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:26.554973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 177/344] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:26.555845 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33ec5-5b47-44f1-9a91-f0dc77e741f0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:26.556611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:26.578361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79c940-5ba6-4183-b698-bff76ea5c5f3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:26.641382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128642c-a965-4a88-8caf-86eff0ccd908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:26.642363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128642c-a965-4a88-8caf-86eff0ccd908 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:26.642780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128642c-a965-4a88-8caf-86eff0ccd908 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:26.643188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128642c-a965-4a88-8caf-86eff0ccd908 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:26.645893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128642c-a965-4a88-8caf-86eff0ccd908 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:26.647518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128642c-a965-4a88-8caf-86eff0ccd908 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:26.650021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 168/345] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:26 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:26.651461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1a861-a03d-430a-8283-7f8866582031 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:26.651897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1a861-a03d-430a-8283-7f8866582031 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:26.652391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1a861-a03d-430a-8283-7f8866582031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:26.668612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33ec5-5b47-44f1-9a91-f0dc77e741f0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:26.686854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f7b1a861-a03d-430a-8283-7f8866582031 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:26.688521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f7b1a861-a03d-430a-8283-7f8866582031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:26.703958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edc0e8-abeb-4133-ae31-480d009b9a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:26.704881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edc0e8-abeb-4133-ae31-480d009b9a42 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:26.704881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edc0e8-abeb-4133-ae31-480d009b9a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:26.705152 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edc0e8-abeb-4133-ae31-480d009b9a42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:26.768470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:26.776762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edc0e8-abeb-4133-ae31-480d009b9a42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:26.778318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edc0e8-abeb-4133-ae31-480d009b9a42 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:26.797226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 169/346] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:26.798297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1a861-a03d-430a-8283-7f8866582031 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:26.904832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:26.920644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1a861-a03d-430a-8283-7f8866582031 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:26.935718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 178/347] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:26.936865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edc0e8-abeb-4133-ae31-480d009b9a42 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:26.968980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaca6d-bcfa-419f-a8fb-2e86069b1852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:26.969856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaca6d-bcfa-419f-a8fb-2e86069b1852 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:26.970377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaca6d-bcfa-419f-a8fb-2e86069b1852 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:26.971744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaca6d-bcfa-419f-a8fb-2e86069b1852 None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:26.984373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaca6d-bcfa-419f-a8fb-2e86069b1852 None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:26.986203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaca6d-bcfa-419f-a8fb-2e86069b1852 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:26.990807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 170/348] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:26 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:26.991737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaca6d-bcfa-419f-a8fb-2e86069b1852 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:27.114591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaca6d-bcfa-419f-a8fb-2e86069b1852 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:27.140467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:27.154942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7d86f-96a1-4e4b-a4bf-ca6fe859aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:27.155854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d86f-96a1-4e4b-a4bf-ca6fe859aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.156301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d86f-96a1-4e4b-a4bf-ca6fe859aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.156717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d86f-96a1-4e4b-a4bf-ca6fe859aaf1 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.157101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edc0e8-abeb-4133-ae31-480d009b9a42 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:27.161238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7d86f-96a1-4e4b-a4bf-ca6fe859aaf1 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:27.162912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7d86f-96a1-4e4b-a4bf-ca6fe859aaf1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:27.165593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 171/349] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:26 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:27.170697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7d86f-96a1-4e4b-a4bf-ca6fe859aaf1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:27.218394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6881a3-7db0-4352-adce-81037612a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:27.220696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6881a3-7db0-4352-adce-81037612a59e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.224778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6881a3-7db0-4352-adce-81037612a59e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.225217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6881a3-7db0-4352-adce-81037612a59e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.226422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:27.250735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6881a3-7db0-4352-adce-81037612a59e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6a6881a3-7db0-4352-adce-81037612a59e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:27.256381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:27.257279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:27.258106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 179/350] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:27.258794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6881a3-7db0-4352-adce-81037612a59e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:27.345055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7d86f-96a1-4e4b-a4bf-ca6fe859aaf1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:27.377900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6881a3-7db0-4352-adce-81037612a59e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:27.387515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:27.388605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.389105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.389577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.390612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] RBAC: Adding query filter params (group.id=83431b76d8834605aa9d7874eb9ec1dd, role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=d38685d4e3e64d9ba662890b9147cac3) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:27.391157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:27.392964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:27.400472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 172/351] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:27 2022] GET /identity/v3/role_assignments?group.id=83431b76d8834605aa9d7874eb9ec1dd&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:27.401544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:27.431831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f595f-a4c0-491b-b2bc-76195fbe0ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:27.432636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f595f-a4c0-491b-b2bc-76195fbe0ccf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.432897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f595f-a4c0-491b-b2bc-76195fbe0ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.433175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f595f-a4c0-491b-b2bc-76195fbe0ccf None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.436793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f595f-a4c0-491b-b2bc-76195fbe0ccf None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:27.437632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f595f-a4c0-491b-b2bc-76195fbe0ccf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:27.439554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 180/352] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:27 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:27.440837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d95ba-dfdd-4da5-b7a6-20f6f55061f0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.441122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d95ba-dfdd-4da5-b7a6-20f6f55061f0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.441952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d95ba-dfdd-4da5-b7a6-20f6f55061f0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.443692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 181/353] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:27.449564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081838c-0b8a-4893-b7bd-a79e4110ed33 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.449890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081838c-0b8a-4893-b7bd-a79e4110ed33 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.450107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081838c-0b8a-4893-b7bd-a79e4110ed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.476321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c081838c-0b8a-4893-b7bd-a79e4110ed33 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:27.477539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c081838c-0b8a-4893-b7bd-a79e4110ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:27.526886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:27.538239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:27.561463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59762d-0859-4940-b896-13978c5ee234 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:27.598073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 182/354] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:27.599289 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081838c-0b8a-4893-b7bd-a79e4110ed33 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:27.618463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1464bad-02eb-42f0-83d6-c813790e2a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:27.620651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464bad-02eb-42f0-83d6-c813790e2a36 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.620935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464bad-02eb-42f0-83d6-c813790e2a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.621212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464bad-02eb-42f0-83d6-c813790e2a36 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.622648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1464bad-02eb-42f0-83d6-c813790e2a36 None admin] RBAC: Adding query filter params (name=member) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:27.622958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1464bad-02eb-42f0-83d6-c813790e2a36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:27.624639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1464bad-02eb-42f0-83d6-c813790e2a36 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:27.640679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 173/355] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:27.641767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c32774-73e1-4959-87f7-0ce2ffa26227 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.642061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c32774-73e1-4959-87f7-0ce2ffa26227 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.642432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c32774-73e1-4959-87f7-0ce2ffa26227 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.643808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 174/356] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:27.645057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c32774-73e1-4959-87f7-0ce2ffa26227 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:27.744025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081838c-0b8a-4893-b7bd-a79e4110ed33 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:27.758463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c32774-73e1-4959-87f7-0ce2ffa26227 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:27.783531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:27.784165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.784442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.784711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.785527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] RBAC: Adding query filter params (group.id=83431b76d8834605aa9d7874eb9ec1dd, role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=d38685d4e3e64d9ba662890b9147cac3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:27.785797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:27.787210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:27.793076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 183/357] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:27 2022] GET /identity/v3/role_assignments?group.id=83431b76d8834605aa9d7874eb9ec1dd&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:27.793939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:27.797840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dab0c9-904f-4b29-aa3c-c6ad6f9ab649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:27.798495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dab0c9-904f-4b29-aa3c-c6ad6f9ab649 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.798773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dab0c9-904f-4b29-aa3c-c6ad6f9ab649 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.799041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dab0c9-904f-4b29-aa3c-c6ad6f9ab649 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.811169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dab0c9-904f-4b29-aa3c-c6ad6f9ab649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-a1dab0c9-904f-4b29-aa3c-c6ad6f9ab649 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:27.817873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:27.818833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:27.819639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 175/358] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:27.821356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf099d7-589e-45bb-ace4-ef42872588e1 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:27.821501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf099d7-589e-45bb-ace4-ef42872588e1 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:27.821642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf099d7-589e-45bb-ace4-ef42872588e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:27.837622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:27.840768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:27.846544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-fdf099d7-589e-45bb-ace4-ef42872588e1 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:27.848700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fdf099d7-589e-45bb-ace4-ef42872588e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:27.947374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 176/359] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:27.948474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf099d7-589e-45bb-ace4-ef42872588e1 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:27.977392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64f8c-9d98-4103-aae5-9bd4afe2f890 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:28.025277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30240eeb-f33e-41de-a133-e9ed4e8e819c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:28.025906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30240eeb-f33e-41de-a133-e9ed4e8e819c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.026184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30240eeb-f33e-41de-a133-e9ed4e8e819c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.026455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30240eeb-f33e-41de-a133-e9ed4e8e819c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.076573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf099d7-589e-45bb-ace4-ef42872588e1 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:28.118656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f55a37-dccf-4973-ad93-1eb945ceb282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:28.119190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f55a37-dccf-4973-ad93-1eb945ceb282 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.119604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f55a37-dccf-4973-ad93-1eb945ceb282 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.119875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f55a37-dccf-4973-ad93-1eb945ceb282 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.120707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f55a37-dccf-4973-ad93-1eb945ceb282 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:28.121012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f55a37-dccf-4973-ad93-1eb945ceb282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:28.122381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f55a37-dccf-4973-ad93-1eb945ceb282 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:28.122727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30240eeb-f33e-41de-a133-e9ed4e8e819c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:28.124384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30240eeb-f33e-41de-a133-e9ed4e8e819c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:28.124915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:28.138956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 177/360] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:28.140452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a3053-3a91-400e-8acd-8b22dc9b2afc None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.140743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a3053-3a91-400e-8acd-8b22dc9b2afc None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.141021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a3053-3a91-400e-8acd-8b22dc9b2afc None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.146765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:28.147505 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 178/361] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:28.149011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a3053-3a91-400e-8acd-8b22dc9b2afc None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:28.257784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 184/362] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:28.258469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30240eeb-f33e-41de-a133-e9ed4e8e819c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:28.276967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a3053-3a91-400e-8acd-8b22dc9b2afc None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:28.313641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4fc4a-2d9c-4585-801c-7d01cf998f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:28.314273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4fc4a-2d9c-4585-801c-7d01cf998f12 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.314554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4fc4a-2d9c-4585-801c-7d01cf998f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.314827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4fc4a-2d9c-4585-801c-7d01cf998f12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.328717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4fc4a-2d9c-4585-801c-7d01cf998f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-00a4fc4a-2d9c-4585-801c-7d01cf998f12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:28.333982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:28.334895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:28.335744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 179/363] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:28.337018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91073014-657a-4aae-ab45-21c92ded7614 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.337179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91073014-657a-4aae-ab45-21c92ded7614 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.337446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91073014-657a-4aae-ab45-21c92ded7614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.356943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-91073014-657a-4aae-ab45-21c92ded7614 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:28.358153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-91073014-657a-4aae-ab45-21c92ded7614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:28.369808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30240eeb-f33e-41de-a133-e9ed4e8e819c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:28.396469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:28.415399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e715987-d48f-4f73-b21a-e582569610d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:28.416043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e715987-d48f-4f73-b21a-e582569610d5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.416434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e715987-d48f-4f73-b21a-e582569610d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.416707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e715987-d48f-4f73-b21a-e582569610d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.419365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:28.438424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 180/364] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:28.439099 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91073014-657a-4aae-ab45-21c92ded7614 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:28.520357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e715987-d48f-4f73-b21a-e582569610d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:28.521829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e715987-d48f-4f73-b21a-e582569610d5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:28.567368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91073014-657a-4aae-ab45-21c92ded7614 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:28.609994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472b0af-d84f-4bb1-a607-6d2e8e7aea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:28.611176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b0af-d84f-4bb1-a607-6d2e8e7aea5a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.611905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b0af-d84f-4bb1-a607-6d2e8e7aea5a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.612555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b0af-d84f-4bb1-a607-6d2e8e7aea5a None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.614664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472b0af-d84f-4bb1-a607-6d2e8e7aea5a None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:28.616367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472b0af-d84f-4bb1-a607-6d2e8e7aea5a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:28.618541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 181/365] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:28 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:28.619512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472b0af-d84f-4bb1-a607-6d2e8e7aea5a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:28.646593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 185/366] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:28.649546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e715987-d48f-4f73-b21a-e582569610d5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:28.673275 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:28.692568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:28.739304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472b0af-d84f-4bb1-a607-6d2e8e7aea5a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:28.775346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:28.776617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.777101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.777359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e715987-d48f-4f73-b21a-e582569610d5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:28.777806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.778675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:28.779175 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:28.780772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:28.790807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 182/367] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:28.791966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:28.834741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1f38c-17e1-40fc-9cb2-0773d65ba421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:28.835208 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1f38c-17e1-40fc-9cb2-0773d65ba421 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.835491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1f38c-17e1-40fc-9cb2-0773d65ba421 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.835756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1f38c-17e1-40fc-9cb2-0773d65ba421 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.848969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1f38c-17e1-40fc-9cb2-0773d65ba421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7ed1f38c-17e1-40fc-9cb2-0773d65ba421 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:28.854413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:28.855396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:28.856237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 186/368] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:28.856934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1f38c-17e1-40fc-9cb2-0773d65ba421 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:28.906025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bcf68-3f0d-4a98-b6a2-90a6fe7200c1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:28.925210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:28.944896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be437b2c-a300-4e24-8031-2063076dfab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:28.945760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be437b2c-a300-4e24-8031-2063076dfab8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:28.946150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be437b2c-a300-4e24-8031-2063076dfab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:28.946523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be437b2c-a300-4e24-8031-2063076dfab8 None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:28.947758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be437b2c-a300-4e24-8031-2063076dfab8 None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:28.949325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be437b2c-a300-4e24-8031-2063076dfab8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:28.949946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:28.951405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 183/369] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:28 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:28.952295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be437b2c-a300-4e24-8031-2063076dfab8 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:28.978476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1f38c-17e1-40fc-9cb2-0773d65ba421 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.016653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ffbf96-83ae-4321-a2a3-6f45eb05952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.017607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffbf96-83ae-4321-a2a3-6f45eb05952f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.018038 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffbf96-83ae-4321-a2a3-6f45eb05952f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.018495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ffbf96-83ae-4321-a2a3-6f45eb05952f None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.020051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ffbf96-83ae-4321-a2a3-6f45eb05952f None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.021844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ffbf96-83ae-4321-a2a3-6f45eb05952f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.023651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 187/370] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:28 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:29.024705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ffbf96-83ae-4321-a2a3-6f45eb05952f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:29.067686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be437b2c-a300-4e24-8031-2063076dfab8 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.104729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9609ef-fd7b-4e7c-98ef-d48a3abdf10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.105475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9609ef-fd7b-4e7c-98ef-d48a3abdf10c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.105823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9609ef-fd7b-4e7c-98ef-d48a3abdf10c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.106171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9609ef-fd7b-4e7c-98ef-d48a3abdf10c None admin] PATH_INFO: `/v3/users/b4036303cbbe4d59a938570ea023bf2e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.107420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9609ef-fd7b-4e7c-98ef-d48a3abdf10c None admin] RBAC: Authorizing `identity:get_user(user_id=b4036303cbbe4d59a938570ea023bf2e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.109028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9609ef-fd7b-4e7c-98ef-d48a3abdf10c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.110780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 184/371] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:15:28 2022] GET /identity/v3/users/b4036303cbbe4d59a938570ea023bf2e => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:29.111631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9609ef-fd7b-4e7c-98ef-d48a3abdf10c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:29.137968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ffbf96-83ae-4321-a2a3-6f45eb05952f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.173276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:29.180600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.181545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.181996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.182439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.183295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:29.183767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.185332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.195594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 188/372] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:29.196742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:29.207660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:29.227702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9609ef-fd7b-4e7c-98ef-d48a3abdf10c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.265367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.266076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.266427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.266765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.267621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] RBAC: Adding query filter params (group.id=83431b76d8834605aa9d7874eb9ec1dd, role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=fa381b3d44664d6ab4beefcf506a841b) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:29.267995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.270187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.276357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 185/373] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:29 2022] GET /identity/v3/role_assignments?group.id=83431b76d8834605aa9d7874eb9ec1dd&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:29.277254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:29.318522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2df85-1d1c-4850-a9ba-9f62e10896d1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.357704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bdced-5458-4322-a79f-90622203ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.358734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bdced-5458-4322-a79f-90622203ac4e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.359234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bdced-5458-4322-a79f-90622203ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.359716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925bdced-5458-4322-a79f-90622203ac4e None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.361067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bdced-5458-4322-a79f-90622203ac4e None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.362899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925bdced-5458-4322-a79f-90622203ac4e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.365424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 189/374] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:29 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:29.366448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bdced-5458-4322-a79f-90622203ac4e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:29.431968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea5e23-a6f4-4690-9e77-2a5d1e07405b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.450105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:29.455218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:29.471166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.471778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.472057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.472382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.473667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] RBAC: Adding query filter params (role.id=a880da5b6bd94a89b2b2658b51f69ee9, user.id=b4036303cbbe4d59a938570ea023bf2e, scope.system=all) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:29.474008 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.475652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.490032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925bdced-5458-4322-a79f-90622203ac4e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.491541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 186/375] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Aug 11 16:15:29 2022] GET /identity/v3/role_assignments?user.id=b4036303cbbe4d59a938570ea023bf2e&scope.system=all&role.id=a880da5b6bd94a89b2b2658b51f69ee9 => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:29.492438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:29.530805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851f8c4-b508-40e5-a8e1-2b571b139b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.531890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851f8c4-b508-40e5-a8e1-2b571b139b26 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.532487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851f8c4-b508-40e5-a8e1-2b571b139b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.532980 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851f8c4-b508-40e5-a8e1-2b571b139b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.606067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a358854-a84e-44d6-9ef8-f0f9d305bf7f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.616698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851f8c4-b508-40e5-a8e1-2b571b139b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.618603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851f8c4-b508-40e5-a8e1-2b571b139b26 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.647206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.647847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.648161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.648405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.649145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] RBAC: Adding query filter params (group.id=83431b76d8834605aa9d7874eb9ec1dd, role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=fa381b3d44664d6ab4beefcf506a841b) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:29.649452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.651041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.657055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 187/376] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:29 2022] GET /identity/v3/role_assignments?group.id=83431b76d8834605aa9d7874eb9ec1dd&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:29.714183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:29.752779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 190/377] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:29.758874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:29.813549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:29.896871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cf4c-726e-4bce-946f-021b07edb104 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:29.939539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faaec35-aaea-43d7-9ace-72b8ee92004f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:29.940203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faaec35-aaea-43d7-9ace-72b8ee92004f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:29.940486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faaec35-aaea-43d7-9ace-72b8ee92004f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:29.940755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faaec35-aaea-43d7-9ace-72b8ee92004f None admin] PATH_INFO: `/v3/projects/307850a266084dc1bb810ff53de8b0aa` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:29.942045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faaec35-aaea-43d7-9ace-72b8ee92004f None admin] RBAC: Authorizing `identity:get_project(project_id=307850a266084dc1bb810ff53de8b0aa)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:29.943481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faaec35-aaea-43d7-9ace-72b8ee92004f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:29.945178 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 188/378] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:29 2022] GET /identity/v3/projects/307850a266084dc1bb810ff53de8b0aa => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:29.950863 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851f8c4-b508-40e5-a8e1-2b571b139b26 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:30.066097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851f8c4-b508-40e5-a8e1-2b571b139b26 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:30.106808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345a153-b6eb-4ed6-9524-b880aa7cec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:30.107444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a153-b6eb-4ed6-9524-b880aa7cec1b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:30.107721 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a153-b6eb-4ed6-9524-b880aa7cec1b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:30.107988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345a153-b6eb-4ed6-9524-b880aa7cec1b None admin] PATH_INFO: `/v3/groups/e8524d4e696743eda1e4456b1623c6eb` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:30.119558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345a153-b6eb-4ed6-9524-b880aa7cec1b None admin] RBAC: Authorizing `identity:get_group(group_id=e8524d4e696743eda1e4456b1623c6eb)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:30.121091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345a153-b6eb-4ed6-9524-b880aa7cec1b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:30.123077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 191/379] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:29 2022] GET /identity/v3/groups/e8524d4e696743eda1e4456b1623c6eb => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:30.129749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faaec35-aaea-43d7-9ace-72b8ee92004f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:30.269024 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faaec35-aaea-43d7-9ace-72b8ee92004f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:30.280366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:30.308745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf953c-1691-4fd7-a801-f0725088e765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:30.309285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf953c-1691-4fd7-a801-f0725088e765 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:30.309562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf953c-1691-4fd7-a801-f0725088e765 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:30.309835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf953c-1691-4fd7-a801-f0725088e765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:30.310581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf953c-1691-4fd7-a801-f0725088e765 None admin] RBAC: Adding query filter params (group.id=e8524d4e696743eda1e4456b1623c6eb, role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=307850a266084dc1bb810ff53de8b0aa) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:30.310892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf953c-1691-4fd7-a801-f0725088e765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:30.312348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf953c-1691-4fd7-a801-f0725088e765 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:30.319479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 189/380] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:30 2022] GET /identity/v3/role_assignments?group.id=e8524d4e696743eda1e4456b1623c6eb&scope.project.id=307850a266084dc1bb810ff53de8b0aa&role.id=29fd031edd2b4f65aed8942853150f92 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:30.347214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47781651-4bbc-4dc2-b793-04c73fb9be1f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:30.348075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47781651-4bbc-4dc2-b793-04c73fb9be1f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:30.348075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47781651-4bbc-4dc2-b793-04c73fb9be1f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:30.349550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 192/381] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:30.356242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b82467-bdc8-4e9a-906c-25c867dcf616 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:30.356545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b82467-bdc8-4e9a-906c-25c867dcf616 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:30.356803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b82467-bdc8-4e9a-906c-25c867dcf616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:30.376266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f7b82467-bdc8-4e9a-906c-25c867dcf616 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:30.377504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f7b82467-bdc8-4e9a-906c-25c867dcf616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:30.435226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9435-aeba-4f77-bf17-b4c9e3de75c9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:30.435874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9435-aeba-4f77-bf17-b4c9e3de75c9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:30.436392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e9435-aeba-4f77-bf17-b4c9e3de75c9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:30.438547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 193/382] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:30.445690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55014385-6d5c-47f5-9cde-9d56ed747c05 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:30.446271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55014385-6d5c-47f5-9cde-9d56ed747c05 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:30.446767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55014385-6d5c-47f5-9cde-9d56ed747c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:30.459484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 190/383] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:30.474730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-55014385-6d5c-47f5-9cde-9d56ed747c05 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:30.476308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-55014385-6d5c-47f5-9cde-9d56ed747c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:30.503059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b82467-bdc8-4e9a-906c-25c867dcf616 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:30.515842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:30.591368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 194/384] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:30.622002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b82467-bdc8-4e9a-906c-25c867dcf616 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:30.623552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:30.627776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55014385-6d5c-47f5-9cde-9d56ed747c05 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:30.658570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677c109-e664-4260-9e07-303db16c0365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:30.659212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677c109-e664-4260-9e07-303db16c0365 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:30.659496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677c109-e664-4260-9e07-303db16c0365 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:30.659779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677c109-e664-4260-9e07-303db16c0365 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:30.686945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677c109-e664-4260-9e07-303db16c0365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-8677c109-e664-4260-9e07-303db16c0365 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:30.692271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:30.693230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:30.694084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 191/385] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:30.699332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677c109-e664-4260-9e07-303db16c0365 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:30.774671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55014385-6d5c-47f5-9cde-9d56ed747c05 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:30.818022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44642b19-45e0-4cfc-b036-c85a04cecf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:30.818663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44642b19-45e0-4cfc-b036-c85a04cecf6b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:30.818944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44642b19-45e0-4cfc-b036-c85a04cecf6b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:30.819217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44642b19-45e0-4cfc-b036-c85a04cecf6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:30.844492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44642b19-45e0-4cfc-b036-c85a04cecf6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-44642b19-45e0-4cfc-b036-c85a04cecf6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:30.849807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:30.850728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:30.851540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 195/386] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:30.857589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44642b19-45e0-4cfc-b036-c85a04cecf6b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:30.902348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:30.942196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677c109-e664-4260-9e07-303db16c0365 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:30.991036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44642b19-45e0-4cfc-b036-c85a04cecf6b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:31.032960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:31.033583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:31.033863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:31.034135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:31.035289 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:31.035887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:31.036379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:31.036646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:31.037323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:31.037615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:31.041627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] RBAC: Adding query filter params (name=member) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:31.041627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:31.041627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:31.043017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:31.056067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 192/387] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:31.062107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 196/388] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:31.064725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:31.073096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:31.123103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:31.183656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:31.196317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b5afd-a490-41cc-a1fd-dc21f4bb8914 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:31.243907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b2896-6646-4274-9959-4651bf3c8ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:31.244599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b2896-6646-4274-9959-4651bf3c8ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:31.244882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b2896-6646-4274-9959-4651bf3c8ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:31.245225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b2896-6646-4274-9959-4651bf3c8ca0 None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:31.247631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b2896-6646-4274-9959-4651bf3c8ca0 None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:31.248819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b2896-6646-4274-9959-4651bf3c8ca0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:31.256919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 197/389] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:31 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:31.262395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b2896-6646-4274-9959-4651bf3c8ca0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:31.314649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef27e9-4b61-4b18-9dd1-440a61732526 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:31.393958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85773d-e722-4c4b-83c1-278184fbe954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:31.394606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85773d-e722-4c4b-83c1-278184fbe954 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:31.394881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85773d-e722-4c4b-83c1-278184fbe954 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:31.395146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85773d-e722-4c4b-83c1-278184fbe954 None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:31.396608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85773d-e722-4c4b-83c1-278184fbe954 None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:31.399507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85773d-e722-4c4b-83c1-278184fbe954 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:31.401134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 193/390] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:31 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:31.406726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85773d-e722-4c4b-83c1-278184fbe954 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:31.437592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b2896-6646-4274-9959-4651bf3c8ca0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:31.478910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afd425-3f64-478b-ae2f-4a0fa65193d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:31.479556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afd425-3f64-478b-ae2f-4a0fa65193d9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:31.479835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afd425-3f64-478b-ae2f-4a0fa65193d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:31.480123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afd425-3f64-478b-ae2f-4a0fa65193d9 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:31.481665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afd425-3f64-478b-ae2f-4a0fa65193d9 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:31.483179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afd425-3f64-478b-ae2f-4a0fa65193d9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:31.485555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 198/391] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:31 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:31.491935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afd425-3f64-478b-ae2f-4a0fa65193d9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:31.560627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:31.587008 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:31.601609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85773d-e722-4c4b-83c1-278184fbe954 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:31.653641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916384d2-65bb-4269-a936-11b3a6f4ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:31.654314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916384d2-65bb-4269-a936-11b3a6f4ba6e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:31.654601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916384d2-65bb-4269-a936-11b3a6f4ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:31.654872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916384d2-65bb-4269-a936-11b3a6f4ba6e None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:31.656213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916384d2-65bb-4269-a936-11b3a6f4ba6e None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:31.657752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916384d2-65bb-4269-a936-11b3a6f4ba6e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:31.659500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 194/392] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:31 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:31.663384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afd425-3f64-478b-ae2f-4a0fa65193d9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:31.666522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916384d2-65bb-4269-a936-11b3a6f4ba6e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:31.711666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e7d5-2282-43f9-90e9-87d238f885ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:31.712377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e7d5-2282-43f9-90e9-87d238f885ec None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:31.712662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e7d5-2282-43f9-90e9-87d238f885ec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:31.712936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e7d5-2282-43f9-90e9-87d238f885ec None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3/groups/83431b76d8834605aa9d7874eb9ec1dd/roles/3b5810bc1bb047f7851cef3afb74dcd0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:31.717847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546e7d5-2282-43f9-90e9-87d238f885ec None admin] RBAC: Authorizing `identity:create_grant(project_id=d38685d4e3e64d9ba662890b9147cac3, group_id=83431b76d8834605aa9d7874eb9ec1dd, role_id=3b5810bc1bb047f7851cef3afb74dcd0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:31.719304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546e7d5-2282-43f9-90e9-87d238f885ec None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:31.744463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 199/393] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Aug 11 16:15:31 2022] PUT /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3/groups/83431b76d8834605aa9d7874eb9ec1dd/roles/3b5810bc1bb047f7851cef3afb74dcd0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:31.802115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916384d2-65bb-4269-a936-11b3a6f4ba6e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:31.848049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a31f9-57e4-4c8b-b6fd-3bf6b639379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:31.848878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a31f9-57e4-4c8b-b6fd-3bf6b639379e None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:31.849256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a31f9-57e4-4c8b-b6fd-3bf6b639379e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:31.849549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a31f9-57e4-4c8b-b6fd-3bf6b639379e None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3/groups/83431b76d8834605aa9d7874eb9ec1dd/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:31.857857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a31f9-57e4-4c8b-b6fd-3bf6b639379e None admin] RBAC: Authorizing `identity:create_grant(project_id=d38685d4e3e64d9ba662890b9147cac3, group_id=83431b76d8834605aa9d7874eb9ec1dd, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:31.859337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a31f9-57e4-4c8b-b6fd-3bf6b639379e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:31.876709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 195/394] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Aug 11 16:15:31 2022] PUT /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3/groups/83431b76d8834605aa9d7874eb9ec1dd/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:32.221673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f9e6-d550-4c8f-ba74-6bd75b69d5b0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:32.222068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f9e6-d550-4c8f-ba74-6bd75b69d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:32.222257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f9e6-d550-4c8f-ba74-6bd75b69d5b0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:32.223817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 200/395] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:32.229210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fcba-d6ba-4fb5-969c-1a63504c3e54 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:32.229499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fcba-d6ba-4fb5-969c-1a63504c3e54 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:32.229775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fcba-d6ba-4fb5-969c-1a63504c3e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:32.257205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-6ae7fcba-d6ba-4fb5-969c-1a63504c3e54 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:32.258413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6ae7fcba-d6ba-4fb5-969c-1a63504c3e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:32.279462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:32.345832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 196/396] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:32.380347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410f9e6-d550-4c8f-ba74-6bd75b69d5b0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:32.477542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6ecbd-2801-4118-a1aa-07c70a536b0f None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:32.477898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6ecbd-2801-4118-a1aa-07c70a536b0f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:32.478085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6ecbd-2801-4118-a1aa-07c70a536b0f None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:32.479430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 197/397] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:32.490124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b34c6-438c-452c-b5d0-797f24bbab02 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:32.490397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b34c6-438c-452c-b5d0-797f24bbab02 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:32.490673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b34c6-438c-452c-b5d0-797f24bbab02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:32.500423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:32.518642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410f9e6-d550-4c8f-ba74-6bd75b69d5b0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:32.529153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-329b34c6-438c-452c-b5d0-797f24bbab02 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:32.530368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-329b34c6-438c-452c-b5d0-797f24bbab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:32.559642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86608bc-a5e3-4bfc-bc58-1984b684304e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:32.560318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86608bc-a5e3-4bfc-bc58-1984b684304e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:32.560597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86608bc-a5e3-4bfc-bc58-1984b684304e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:32.560869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86608bc-a5e3-4bfc-bc58-1984b684304e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:32.574492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86608bc-a5e3-4bfc-bc58-1984b684304e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f86608bc-a5e3-4bfc-bc58-1984b684304e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:32.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:32.580319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:32.581137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 201/398] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:32.587136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86608bc-a5e3-4bfc-bc58-1984b684304e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:32.669016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 198/399] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:32.693241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b34c6-438c-452c-b5d0-797f24bbab02 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:32.737096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86608bc-a5e3-4bfc-bc58-1984b684304e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:32.775487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:32.776122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:32.776409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:32.776692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:32.777354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:32.777654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:32.779029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:32.789450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 202/400] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:32.790890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:32.797169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:32.817583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b34c6-438c-452c-b5d0-797f24bbab02 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:32.819255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:32.855259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba27c7e-72b6-4369-990e-da890da5bf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:32.856020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba27c7e-72b6-4369-990e-da890da5bf92 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:32.856375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba27c7e-72b6-4369-990e-da890da5bf92 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:32.856714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba27c7e-72b6-4369-990e-da890da5bf92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:32.868657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba27c7e-72b6-4369-990e-da890da5bf92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-7ba27c7e-72b6-4369-990e-da890da5bf92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:32.873411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:32.874290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:32.875123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 199/401] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:32.880760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba27c7e-72b6-4369-990e-da890da5bf92 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:32.920540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84039a4-28d8-4cc3-ae57-f7217c4ec25f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:32.964866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32126701-33cd-4d22-b57c-0767ceb08ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:32.965501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32126701-33cd-4d22-b57c-0767ceb08ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:32.965774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32126701-33cd-4d22-b57c-0767ceb08ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:32.966041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32126701-33cd-4d22-b57c-0767ceb08ae3 None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:32.967239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32126701-33cd-4d22-b57c-0767ceb08ae3 None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:32.969342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32126701-33cd-4d22-b57c-0767ceb08ae3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:32.971391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 203/402] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:32 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:32.981055 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32126701-33cd-4d22-b57c-0767ceb08ae3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:33.033862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba27c7e-72b6-4369-990e-da890da5bf92 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:33.100503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:33.101110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.101386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.101651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.102316 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:33.102612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:33.103983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:33.119504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 200/403] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:33.136265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:33.137101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:33.148331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:33.183879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32126701-33cd-4d22-b57c-0767ceb08ae3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:33.258227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c4a47-de56-4f90-9443-8afa9c7800ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:33.258876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4a47-de56-4f90-9443-8afa9c7800ed None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.259155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4a47-de56-4f90-9443-8afa9c7800ed None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.259428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4a47-de56-4f90-9443-8afa9c7800ed None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.264897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c4a47-de56-4f90-9443-8afa9c7800ed None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:33.266380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c4a47-de56-4f90-9443-8afa9c7800ed None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:33.268445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 204/404] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:32 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:33.282297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c4a47-de56-4f90-9443-8afa9c7800ed None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:33.288950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3e38d-2e25-4d43-b785-23f99470132a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:33.327816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389b725-2c5a-4561-b9e4-11f21a7a7a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:33.328474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b725-2c5a-4561-b9e4-11f21a7a7a1a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.328760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b725-2c5a-4561-b9e4-11f21a7a7a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.329022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389b725-2c5a-4561-b9e4-11f21a7a7a1a None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.330424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389b725-2c5a-4561-b9e4-11f21a7a7a1a None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:33.332765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389b725-2c5a-4561-b9e4-11f21a7a7a1a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:33.334407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 201/405] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:33 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:33.335532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ace1a-7a43-41b4-8330-469774b9cbb1 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.335808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ace1a-7a43-41b4-8330-469774b9cbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.336078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ace1a-7a43-41b4-8330-469774b9cbb1 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.337551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 202/406] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:33.342931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ace1a-7a43-41b4-8330-469774b9cbb1 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:33.440995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c4a47-de56-4f90-9443-8afa9c7800ed None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:33.442273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:33.481005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f7ba5-38bd-4022-abd2-18df3e75214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:33.481670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f7ba5-38bd-4022-abd2-18df3e75214d None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.481949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f7ba5-38bd-4022-abd2-18df3e75214d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.482076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ace1a-7a43-41b4-8330-469774b9cbb1 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:33.482265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f7ba5-38bd-4022-abd2-18df3e75214d None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b/groups/83431b76d8834605aa9d7874eb9ec1dd/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.485102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f7ba5-38bd-4022-abd2-18df3e75214d None admin] RBAC: Authorizing `identity:create_grant(project_id=fa381b3d44664d6ab4beefcf506a841b, group_id=83431b76d8834605aa9d7874eb9ec1dd, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:33.486983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f7ba5-38bd-4022-abd2-18df3e75214d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:33.507033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 205/407] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Aug 11 16:15:33 2022] PUT /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b/groups/83431b76d8834605aa9d7874eb9ec1dd/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:15:33.508790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac9c1b-bcc9-4cfd-b207-8f93493a52ee None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.509078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac9c1b-bcc9-4cfd-b207-8f93493a52ee None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.509404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac9c1b-bcc9-4cfd-b207-8f93493a52ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.517306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:33.533105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-68ac9c1b-bcc9-4cfd-b207-8f93493a52ee None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:33.535495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-68ac9c1b-bcc9-4cfd-b207-8f93493a52ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:33.543180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433e0f3-e286-44c5-9b9c-ad9a32fba211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:33.543981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433e0f3-e286-44c5-9b9c-ad9a32fba211 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.544409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433e0f3-e286-44c5-9b9c-ad9a32fba211 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.544786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433e0f3-e286-44c5-9b9c-ad9a32fba211 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.546155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433e0f3-e286-44c5-9b9c-ad9a32fba211 None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:33.547718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433e0f3-e286-44c5-9b9c-ad9a32fba211 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:33.553375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 203/408] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:33 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:33.557023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433e0f3-e286-44c5-9b9c-ad9a32fba211 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:33.656505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 206/409] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:33.697724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac9c1b-bcc9-4cfd-b207-8f93493a52ee None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:33.741937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433e0f3-e286-44c5-9b9c-ad9a32fba211 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:33.787641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95cbe2-0973-4f7d-b43c-a5646101376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:33.789421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95cbe2-0973-4f7d-b43c-a5646101376a None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.789703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95cbe2-0973-4f7d-b43c-a5646101376a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.789981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95cbe2-0973-4f7d-b43c-a5646101376a None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b/groups/83431b76d8834605aa9d7874eb9ec1dd/roles/3b5810bc1bb047f7851cef3afb74dcd0` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.792542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95cbe2-0973-4f7d-b43c-a5646101376a None admin] RBAC: Authorizing `identity:create_grant(project_id=fa381b3d44664d6ab4beefcf506a841b, group_id=83431b76d8834605aa9d7874eb9ec1dd, role_id=3b5810bc1bb047f7851cef3afb74dcd0)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:33.793984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95cbe2-0973-4f7d-b43c-a5646101376a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:33.804657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:33.811451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 204/410] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Aug 11 16:15:33 2022] PUT /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b/groups/83431b76d8834605aa9d7874eb9ec1dd/roles/3b5810bc1bb047f7851cef3afb74dcd0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:15:33.841398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac9c1b-bcc9-4cfd-b207-8f93493a52ee None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:33.884251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d660ef-1860-4ef4-91f9-8522822ea3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:33.884922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d660ef-1860-4ef4-91f9-8522822ea3db None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:33.885200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d660ef-1860-4ef4-91f9-8522822ea3db None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:33.885473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d660ef-1860-4ef4-91f9-8522822ea3db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:33.887683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:33.900732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d660ef-1860-4ef4-91f9-8522822ea3db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-98d660ef-1860-4ef4-91f9-8522822ea3db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:33.907849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:33.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:33.909644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 207/411] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:33.916533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95cbe2-0973-4f7d-b43c-a5646101376a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:34.055770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95cbe2-0973-4f7d-b43c-a5646101376a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:34.095094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:34.095716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.095994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.096310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.096985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:34.097330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:34.099246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:34.112242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 205/412] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:34.114469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d660ef-1860-4ef4-91f9-8522822ea3db None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:34.229026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d660ef-1860-4ef4-91f9-8522822ea3db None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:34.271714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a089fc-d400-4262-8bad-17bcf393b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:34.272429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a089fc-d400-4262-8bad-17bcf393b382 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.272706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a089fc-d400-4262-8bad-17bcf393b382 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.272978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a089fc-d400-4262-8bad-17bcf393b382 None admin] PATH_INFO: `/v3/groups/e8524d4e696743eda1e4456b1623c6eb` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.276766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a089fc-d400-4262-8bad-17bcf393b382 None admin] RBAC: Authorizing `identity:get_group(group_id=e8524d4e696743eda1e4456b1623c6eb)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:34.278217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a089fc-d400-4262-8bad-17bcf393b382 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:34.279998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 208/413] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:34 2022] GET /identity/v3/groups/e8524d4e696743eda1e4456b1623c6eb => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:34.286252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:34.355551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:34.399304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebc832-dbf6-48fc-bea1-fb936566ca95 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.399759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebc832-dbf6-48fc-bea1-fb936566ca95 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.400160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebc832-dbf6-48fc-bea1-fb936566ca95 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.401473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac661ec8-7d23-4350-af8b-f923c18ea0b5 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:34.404774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 209/414] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:34.408525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8be1e-51db-4a8e-8f06-d89e1c3fe6b4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.408932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8be1e-51db-4a8e-8f06-d89e1c3fe6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.409292 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8be1e-51db-4a8e-8f06-d89e1c3fe6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.431432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-18a8be1e-51db-4a8e-8f06-d89e1c3fe6b4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:34.433273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-18a8be1e-51db-4a8e-8f06-d89e1c3fe6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:34.475422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37f76-e62a-4c6a-ad42-7e7bae32f51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:34.476068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37f76-e62a-4c6a-ad42-7e7bae32f51d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.476370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37f76-e62a-4c6a-ad42-7e7bae32f51d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.476642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37f76-e62a-4c6a-ad42-7e7bae32f51d None admin] PATH_INFO: `/v3/projects/307850a266084dc1bb810ff53de8b0aa` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.480022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c37f76-e62a-4c6a-ad42-7e7bae32f51d None admin] RBAC: Authorizing `identity:get_project(project_id=307850a266084dc1bb810ff53de8b0aa)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:34.484260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c37f76-e62a-4c6a-ad42-7e7bae32f51d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:34.488950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 206/415] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:34 2022] GET /identity/v3/projects/307850a266084dc1bb810ff53de8b0aa => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:34.496276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37f76-e62a-4c6a-ad42-7e7bae32f51d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:34.527561 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:34.549412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 210/416] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:34.577311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8be1e-51db-4a8e-8f06-d89e1c3fe6b4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:34.669680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37f76-e62a-4c6a-ad42-7e7bae32f51d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:34.713769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8be1e-51db-4a8e-8f06-d89e1c3fe6b4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:34.720605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:34.733865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f06175-8793-4115-b22d-536fe9bfb51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:34.734805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f06175-8793-4115-b22d-536fe9bfb51b None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.736625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f06175-8793-4115-b22d-536fe9bfb51b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.737124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f06175-8793-4115-b22d-536fe9bfb51b None admin] PATH_INFO: `/v3/projects/307850a266084dc1bb810ff53de8b0aa/groups/e8524d4e696743eda1e4456b1623c6eb/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.740740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f06175-8793-4115-b22d-536fe9bfb51b None admin] RBAC: Authorizing `identity:create_grant(project_id=307850a266084dc1bb810ff53de8b0aa, group_id=e8524d4e696743eda1e4456b1623c6eb, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:34.742394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f06175-8793-4115-b22d-536fe9bfb51b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:34.763185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 207/417] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Aug 11 16:15:34 2022] PUT /identity/v3/projects/307850a266084dc1bb810ff53de8b0aa/groups/e8524d4e696743eda1e4456b1623c6eb/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:15:34.773067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90f2c-4bfb-49fa-906f-77d58b499aa1 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.773781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90f2c-4bfb-49fa-906f-77d58b499aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.774257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd90f2c-4bfb-49fa-906f-77d58b499aa1 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.775618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3658902-dbe9-4d16-9c0d-6d21acea96ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:34.776359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3658902-dbe9-4d16-9c0d-6d21acea96ce None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.776598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3658902-dbe9-4d16-9c0d-6d21acea96ce None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.776960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3658902-dbe9-4d16-9c0d-6d21acea96ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.777161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 208/418] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:34.783196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd325d4-efc7-41bd-8292-6b16992309fe None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:34.783493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd325d4-efc7-41bd-8292-6b16992309fe None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:34.783743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd325d4-efc7-41bd-8292-6b16992309fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:34.796225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3658902-dbe9-4d16-9c0d-6d21acea96ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d3658902-dbe9-4d16-9c0d-6d21acea96ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:34.803442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:34.804437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:34.805245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:34.805856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-edd325d4-efc7-41bd-8292-6b16992309fe None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:34.805856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-edd325d4-efc7-41bd-8292-6b16992309fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:34.808322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 211/419] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:34.820766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3658902-dbe9-4d16-9c0d-6d21acea96ce None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:34.936072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:34.943978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 209/420] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:34.970681 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd325d4-efc7-41bd-8292-6b16992309fe None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:35.004942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3658902-dbe9-4d16-9c0d-6d21acea96ce None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:35.061503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:35.063059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:35.064442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:35.065100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:35.067101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:35.068337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:35.070485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:35.087235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 212/421] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:35.093168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:35.145933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd325d4-efc7-41bd-8292-6b16992309fe None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:35.178053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:35.212302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49e7c-2584-410b-a53a-ae2f21ba7642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:35.212910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49e7c-2584-410b-a53a-ae2f21ba7642 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:35.213196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49e7c-2584-410b-a53a-ae2f21ba7642 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:35.213468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49e7c-2584-410b-a53a-ae2f21ba7642 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:35.234779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c49e7c-2584-410b-a53a-ae2f21ba7642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-39c49e7c-2584-410b-a53a-ae2f21ba7642 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:35.241051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:35.241962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:35.242781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 210/422] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:35.248504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49e7c-2584-410b-a53a-ae2f21ba7642 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:35.283213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b297f-4e7f-4e62-94b7-3e014c48251f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:35.310420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:35.351070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d16b2-43aa-4fb3-b0cd-b35869eef01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:35.351690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d16b2-43aa-4fb3-b0cd-b35869eef01a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:35.351973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d16b2-43aa-4fb3-b0cd-b35869eef01a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:35.352269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d16b2-43aa-4fb3-b0cd-b35869eef01a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:35.426904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49e7c-2584-410b-a53a-ae2f21ba7642 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:35.462599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:35.463525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:35.464024 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:35.464528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:35.465395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:35.465917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:35.467476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:35.470430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d16b2-43aa-4fb3-b0cd-b35869eef01a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:35.471883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d16b2-43aa-4fb3-b0cd-b35869eef01a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:35.477211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 211/423] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:35.485012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:35.529277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:35.590168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9086a-1c55-406e-8bf8-928923fa3ad6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:35.628982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba9678-3422-4487-bad6-912816f10b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:35.629898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba9678-3422-4487-bad6-912816f10b75 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:35.630386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba9678-3422-4487-bad6-912816f10b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:35.631037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba9678-3422-4487-bad6-912816f10b75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:35.707145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba9678-3422-4487-bad6-912816f10b75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:35.709446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba9678-3422-4487-bad6-912816f10b75 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:35.761798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:35.774855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 213/424] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:35.781063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d16b2-43aa-4fb3-b0cd-b35869eef01a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:35.820834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 212/425] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:35.827149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba9678-3422-4487-bad6-912816f10b75 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:35.829293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:35.938603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba9678-3422-4487-bad6-912816f10b75 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:35.978505 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f79d03-9886-4d6a-b7c3-d5543c252893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:35.979668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79d03-9886-4d6a-b7c3-d5543c252893 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:35.980632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79d03-9886-4d6a-b7c3-d5543c252893 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:35.981376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f79d03-9886-4d6a-b7c3-d5543c252893 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:35.982914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f79d03-9886-4d6a-b7c3-d5543c252893 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:35.985381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f79d03-9886-4d6a-b7c3-d5543c252893 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:35.988533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 213/426] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:35 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:35.995126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f79d03-9886-4d6a-b7c3-d5543c252893 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:36.008202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:36.069547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d16b2-43aa-4fb3-b0cd-b35869eef01a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:36.116891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f79d03-9886-4d6a-b7c3-d5543c252893 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:36.149374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3e540-9dd8-41ca-9aa9-f63d50c49403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:36.149997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3e540-9dd8-41ca-9aa9-f63d50c49403 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.150271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3e540-9dd8-41ca-9aa9-f63d50c49403 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.150539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3e540-9dd8-41ca-9aa9-f63d50c49403 None admin] PATH_INFO: `/v3/projects/d38685d4e3e64d9ba662890b9147cac3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.151760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3e540-9dd8-41ca-9aa9-f63d50c49403 None admin] RBAC: Authorizing `identity:get_project(project_id=d38685d4e3e64d9ba662890b9147cac3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:36.157432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3e540-9dd8-41ca-9aa9-f63d50c49403 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:36.162033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 214/427] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:35 2022] GET /identity/v3/projects/d38685d4e3e64d9ba662890b9147cac3 => generated 298 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:36.164235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d843-2682-4038-8ce3-6d528ee22532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:36.165100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d843-2682-4038-8ce3-6d528ee22532 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.165628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d843-2682-4038-8ce3-6d528ee22532 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.166628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3e540-9dd8-41ca-9aa9-f63d50c49403 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:36.167120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d843-2682-4038-8ce3-6d528ee22532 None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.169106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1d843-2682-4038-8ce3-6d528ee22532 None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:36.172767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1d843-2682-4038-8ce3-6d528ee22532 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:36.176877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 214/428] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:35 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:36.182763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d843-2682-4038-8ce3-6d528ee22532 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:36.280596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:36.303939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:36.312714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3e540-9dd8-41ca-9aa9-f63d50c49403 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:36.330234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d843-2682-4038-8ce3-6d528ee22532 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:36.358695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce0dfcb-16f2-436e-b303-59968340e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:36.359579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0dfcb-16f2-436e-b303-59968340e9bd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.360011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0dfcb-16f2-436e-b303-59968340e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.360442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce0dfcb-16f2-436e-b303-59968340e9bd None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.361689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0dfcb-16f2-436e-b303-59968340e9bd None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:36.364080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce0dfcb-16f2-436e-b303-59968340e9bd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:36.370860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 215/429] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:36 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:36.371426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a855c8-f30c-40bc-b0cb-856a31cd5a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:36.372167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a855c8-f30c-40bc-b0cb-856a31cd5a6d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.372553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a855c8-f30c-40bc-b0cb-856a31cd5a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.372727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a5b6d-2bef-47a8-a38d-f5faff97e630 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.373025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a855c8-f30c-40bc-b0cb-856a31cd5a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.373296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a5b6d-2bef-47a8-a38d-f5faff97e630 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.373697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a5b6d-2bef-47a8-a38d-f5faff97e630 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.374044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a855c8-f30c-40bc-b0cb-856a31cd5a6d None admin] RBAC: Adding query filter params (group.id=83431b76d8834605aa9d7874eb9ec1dd, role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=d38685d4e3e64d9ba662890b9147cac3) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:36.374455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a855c8-f30c-40bc-b0cb-856a31cd5a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:36.375421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 216/430] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:36.376080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a855c8-f30c-40bc-b0cb-856a31cd5a6d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:36.376405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a5b6d-2bef-47a8-a38d-f5faff97e630 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:36.383804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 215/431] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:36 2022] GET /identity/v3/role_assignments?group.id=83431b76d8834605aa9d7874eb9ec1dd&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:36.386468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5955a-d815-44b4-a3fe-ee79dc57cfec None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.386865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5955a-d815-44b4-a3fe-ee79dc57cfec None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.387246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5955a-d815-44b4-a3fe-ee79dc57cfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.410466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e8b5955a-d815-44b4-a3fe-ee79dc57cfec None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:36.412186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e8b5955a-d815-44b4-a3fe-ee79dc57cfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:36.520128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 216/432] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:36.520128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9ef08-48f2-4cea-aff4-5fc6ae1c8a06 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.520128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9ef08-48f2-4cea-aff4-5fc6ae1c8a06 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.520128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9ef08-48f2-4cea-aff4-5fc6ae1c8a06 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.520128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 217/433] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:36.533147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a5b6d-2bef-47a8-a38d-f5faff97e630 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:36.535832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a91f2-d7e3-4fe4-a81d-0a2352d68f35 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.536110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a91f2-d7e3-4fe4-a81d-0a2352d68f35 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.536392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a91f2-d7e3-4fe4-a81d-0a2352d68f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.559378 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-844a91f2-d7e3-4fe4-a81d-0a2352d68f35 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:36.561022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-844a91f2-d7e3-4fe4-a81d-0a2352d68f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:36.585812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:36.586343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.586623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.586896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.587626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] RBAC: Adding query filter params (group.id=83431b76d8834605aa9d7874eb9ec1dd, role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=d38685d4e3e64d9ba662890b9147cac3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:36.587935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:36.589402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:36.596896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 217/434] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:36 2022] GET /identity/v3/role_assignments?group.id=83431b76d8834605aa9d7874eb9ec1dd&scope.project.id=d38685d4e3e64d9ba662890b9147cac3&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:36.597751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:36.616770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:36.646851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:36.651999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 218/435] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:36.679539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a91f2-d7e3-4fe4-a81d-0a2352d68f35 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:36.733717 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5407a4a-ad09-4138-8fd8-e4d92b0d0f9c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:36.784176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb4e07-a325-4352-b061-62dd7c925435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:36.785381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4e07-a325-4352-b061-62dd7c925435 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.786176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4e07-a325-4352-b061-62dd7c925435 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.786176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4e07-a325-4352-b061-62dd7c925435 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.797361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb4e07-a325-4352-b061-62dd7c925435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-14bb4e07-a325-4352-b061-62dd7c925435 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:36.803867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:36.804867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:36.806607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 218/436] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:36.811754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb4e07-a325-4352-b061-62dd7c925435 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:36.817949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a91f2-d7e3-4fe4-a81d-0a2352d68f35 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:36.861170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0795a3-e196-458e-9f5e-442babf96c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:36.862734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0795a3-e196-458e-9f5e-442babf96c8c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:36.862734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0795a3-e196-458e-9f5e-442babf96c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:36.862734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0795a3-e196-458e-9f5e-442babf96c8c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:36.876428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0795a3-e196-458e-9f5e-442babf96c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-ff0795a3-e196-458e-9f5e-442babf96c8c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:36.881425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:36.882370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:36.883196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 219/437] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:36.890046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0795a3-e196-458e-9f5e-442babf96c8c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:36.920083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:36.959854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb4e07-a325-4352-b061-62dd7c925435 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:36.978663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.009174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.009813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.010092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.010353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.011015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:37.011312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.012788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.022935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 219/438] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:15:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:37.028591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.035374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0795a3-e196-458e-9f5e-442babf96c8c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.078593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e719d1-9daf-497e-821c-3fdc478e4890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.079488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e719d1-9daf-497e-821c-3fdc478e4890 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.080444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e719d1-9daf-497e-821c-3fdc478e4890 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.080859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e719d1-9daf-497e-821c-3fdc478e4890 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.082807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e719d1-9daf-497e-821c-3fdc478e4890 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:37.083487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e719d1-9daf-497e-821c-3fdc478e4890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.085415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e719d1-9daf-497e-821c-3fdc478e4890 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.094690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 220/439] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:37.095898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05730397-1529-411c-aade-0b65cc32df71 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.096294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05730397-1529-411c-aade-0b65cc32df71 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.096525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05730397-1529-411c-aade-0b65cc32df71 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.098475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 221/440] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:37.102391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05730397-1529-411c-aade-0b65cc32df71 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.154005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1c7d8-7902-49b3-9256-45d6390323fa None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.193752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f295775-d5a4-48cd-a0d2-dbb2897c3f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.194367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f295775-d5a4-48cd-a0d2-dbb2897c3f02 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.194651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f295775-d5a4-48cd-a0d2-dbb2897c3f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.194919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f295775-d5a4-48cd-a0d2-dbb2897c3f02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.198994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.226799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05730397-1529-411c-aade-0b65cc32df71 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.252416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.265738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0b71b-1550-4a5d-ae99-634269f00480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.266345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0b71b-1550-4a5d-ae99-634269f00480 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.266610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0b71b-1550-4a5d-ae99-634269f00480 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.266876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0b71b-1550-4a5d-ae99-634269f00480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.274392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f295775-d5a4-48cd-a0d2-dbb2897c3f02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.275847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f295775-d5a4-48cd-a0d2-dbb2897c3f02 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.336783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0b71b-1550-4a5d-ae99-634269f00480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.338257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d0b71b-1550-4a5d-ae99-634269f00480 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.374695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 220/441] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 16:15:37.376018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d69e0-f1ba-4976-ae43-ca5e7aa73e8e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.376344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d69e0-f1ba-4976-ae43-ca5e7aa73e8e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.377011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d69e0-f1ba-4976-ae43-ca5e7aa73e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.396018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-af4d69e0-f1ba-4976-ae43-ca5e7aa73e8e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:37.397239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-af4d69e0-f1ba-4976-ae43-ca5e7aa73e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:37.424939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.437549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 222/442] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 16:15:37.438666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0b71b-1550-4a5d-ae99-634269f00480 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.464171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.467787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 221/443] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:37.468540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d69e0-f1ba-4976-ae43-ca5e7aa73e8e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.539411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0b71b-1550-4a5d-ae99-634269f00480 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.568917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d69e0-f1ba-4976-ae43-ca5e7aa73e8e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.572989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36532eb4-945b-41f6-9a82-612d447ef5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.573601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36532eb4-945b-41f6-9a82-612d447ef5de None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.573874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36532eb4-945b-41f6-9a82-612d447ef5de None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.574141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36532eb4-945b-41f6-9a82-612d447ef5de None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.575248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36532eb4-945b-41f6-9a82-612d447ef5de None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.576721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36532eb4-945b-41f6-9a82-612d447ef5de None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.578354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 223/444] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:37.579094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36532eb4-945b-41f6-9a82-612d447ef5de None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.606928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bad89-d94b-43bb-a04c-c48e19d5c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.607549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bad89-d94b-43bb-a04c-c48e19d5c276 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.607833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bad89-d94b-43bb-a04c-c48e19d5c276 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.608122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bad89-d94b-43bb-a04c-c48e19d5c276 None admin] PATH_INFO: `/v3/projects/fa381b3d44664d6ab4beefcf506a841b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.609237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bad89-d94b-43bb-a04c-c48e19d5c276 None admin] RBAC: Authorizing `identity:get_project(project_id=fa381b3d44664d6ab4beefcf506a841b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.610958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bad89-d94b-43bb-a04c-c48e19d5c276 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.617711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 222/445] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/projects/fa381b3d44664d6ab4beefcf506a841b => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:37.617711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bad89-d94b-43bb-a04c-c48e19d5c276 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.646777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.668997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.679818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36532eb4-945b-41f6-9a82-612d447ef5de None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.714125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affa53d-ec7d-47ed-862f-d47613e7352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.714617 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bad89-d94b-43bb-a04c-c48e19d5c276 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.714754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affa53d-ec7d-47ed-862f-d47613e7352d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.715035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affa53d-ec7d-47ed-862f-d47613e7352d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.715306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affa53d-ec7d-47ed-862f-d47613e7352d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.725524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affa53d-ec7d-47ed-862f-d47613e7352d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-7affa53d-ec7d-47ed-862f-d47613e7352d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:37.730295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:37.731163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:37.731978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 224/446] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:37.733327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affa53d-ec7d-47ed-862f-d47613e7352d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.750549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2855fa-2583-4b3d-92c6-b43eb087bd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.751176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2855fa-2583-4b3d-92c6-b43eb087bd1f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.751455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2855fa-2583-4b3d-92c6-b43eb087bd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.751724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2855fa-2583-4b3d-92c6-b43eb087bd1f None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.752873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2855fa-2583-4b3d-92c6-b43eb087bd1f None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.754890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2855fa-2583-4b3d-92c6-b43eb087bd1f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.756640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 223/447] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:37.757249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2855fa-2583-4b3d-92c6-b43eb087bd1f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.836083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affa53d-ec7d-47ed-862f-d47613e7352d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.854488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2855fa-2583-4b3d-92c6-b43eb087bd1f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:37.855998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.870897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790082f3-890a-4529-a222-aab88a21e347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.871525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790082f3-890a-4529-a222-aab88a21e347 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.871804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790082f3-890a-4529-a222-aab88a21e347 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.872077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790082f3-890a-4529-a222-aab88a21e347 None admin] PATH_INFO: `/v3/groups/83431b76d8834605aa9d7874eb9ec1dd` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.873240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790082f3-890a-4529-a222-aab88a21e347 None admin] RBAC: Authorizing `identity:get_group(group_id=83431b76d8834605aa9d7874eb9ec1dd)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.874669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790082f3-890a-4529-a222-aab88a21e347 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.876265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 225/448] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/groups/83431b76d8834605aa9d7874eb9ec1dd => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:37.877015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790082f3-890a-4529-a222-aab88a21e347 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.891484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:37.892085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:37.892395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:37.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:37.893328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:37.893631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:37.895334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:37.896009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:37.905604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 224/449] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:37.906395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:37.979354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790082f3-890a-4529-a222-aab88a21e347 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:38.002703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79158652-fe4a-484f-9b03-986bd0c7c6e8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:38.012473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:38.013068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:38.013341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:38.013608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:38.014323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] RBAC: Adding query filter params (group.id=83431b76d8834605aa9d7874eb9ec1dd, role.id=6fb07818f6ff488d9b09556ed4f1d120, scope.project.id=fa381b3d44664d6ab4beefcf506a841b) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:38.014624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:38.016030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:38.021972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 226/450] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/role_assignments?group.id=83431b76d8834605aa9d7874eb9ec1dd&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=6fb07818f6ff488d9b09556ed4f1d120 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:38.022725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:38.038579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:38.039214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:38.039496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:38.039772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:38.040575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] RBAC: Adding query filter params (group.id=83431b76d8834605aa9d7874eb9ec1dd, role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=fa381b3d44664d6ab4beefcf506a841b) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:38.040886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:38.042383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:38.051817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 225/451] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:37 2022] GET /identity/v3/role_assignments?group.id=83431b76d8834605aa9d7874eb9ec1dd&scope.project.id=fa381b3d44664d6ab4beefcf506a841b&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:38.136057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:38.159293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d86c4-7a08-4ecf-8b88-47df880089ba None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:38.200340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1d13b-0ce1-47f1-bd4d-c0e0ae3cfe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:38.200960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d13b-0ce1-47f1-bd4d-c0e0ae3cfe29 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:38.201237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d13b-0ce1-47f1-bd4d-c0e0ae3cfe29 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:38.201504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d13b-0ce1-47f1-bd4d-c0e0ae3cfe29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:38.279525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1d13b-0ce1-47f1-bd4d-c0e0ae3cfe29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:38.280948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1d13b-0ce1-47f1-bd4d-c0e0ae3cfe29 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:38.363470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:38.383427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 227/452] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:38.389333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:38.419969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:38.494639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09f0bd-459b-4038-8d20-e4d44f0f93ee None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:38.527772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087a62b-cf9d-4d61-84c9-f9eede1d8ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:38.528425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087a62b-cf9d-4d61-84c9-f9eede1d8ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:38.528702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087a62b-cf9d-4d61-84c9-f9eede1d8ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:38.528971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087a62b-cf9d-4d61-84c9-f9eede1d8ca0 None admin] PATH_INFO: `/v3/projects/307850a266084dc1bb810ff53de8b0aa` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:38.530140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087a62b-cf9d-4d61-84c9-f9eede1d8ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=307850a266084dc1bb810ff53de8b0aa)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:38.531516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087a62b-cf9d-4d61-84c9-f9eede1d8ca0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:38.533217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 226/453] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:15:38 2022] GET /identity/v3/projects/307850a266084dc1bb810ff53de8b0aa => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:38.539307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1d13b-0ce1-47f1-bd4d-c0e0ae3cfe29 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:38.637290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1d13b-0ce1-47f1-bd4d-c0e0ae3cfe29 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:38.670534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129f5e4-0871-4db6-ac18-a3503ed4c393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:38.671147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129f5e4-0871-4db6-ac18-a3503ed4c393 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:38.671429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129f5e4-0871-4db6-ac18-a3503ed4c393 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:38.671702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129f5e4-0871-4db6-ac18-a3503ed4c393 None admin] PATH_INFO: `/v3/groups/e8524d4e696743eda1e4456b1623c6eb` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:38.672826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129f5e4-0871-4db6-ac18-a3503ed4c393 None admin] RBAC: Authorizing `identity:get_group(group_id=e8524d4e696743eda1e4456b1623c6eb)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:38.674259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129f5e4-0871-4db6-ac18-a3503ed4c393 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:38.675856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 228/454] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Aug 11 16:15:38 2022] GET /identity/v3/groups/e8524d4e696743eda1e4456b1623c6eb => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:38.681737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087a62b-cf9d-4d61-84c9-f9eede1d8ca0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:38.773281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:38.779241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087a62b-cf9d-4d61-84c9-f9eede1d8ca0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:38.811935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff24537-727a-4e9b-93ef-b42ba8b522af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:38.812553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24537-727a-4e9b-93ef-b42ba8b522af None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:38.812822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24537-727a-4e9b-93ef-b42ba8b522af None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:38.813090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff24537-727a-4e9b-93ef-b42ba8b522af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:38.814057 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff24537-727a-4e9b-93ef-b42ba8b522af None admin] RBAC: Adding query filter params (group.id=e8524d4e696743eda1e4456b1623c6eb, role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=307850a266084dc1bb810ff53de8b0aa) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:38.814361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff24537-727a-4e9b-93ef-b42ba8b522af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:38.815771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff24537-727a-4e9b-93ef-b42ba8b522af None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:38.821306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 227/455] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Aug 11 16:15:38 2022] GET /identity/v3/role_assignments?group.id=e8524d4e696743eda1e4456b1623c6eb&scope.project.id=307850a266084dc1bb810ff53de8b0aa&role.id=29fd031edd2b4f65aed8942853150f92 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:41.265499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b181c-d52a-4391-a43c-b9219a6e449f None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.266042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b181c-d52a-4391-a43c-b9219a6e449f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.266042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b181c-d52a-4391-a43c-b9219a6e449f None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.267377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 229/456] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:41.273149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24961337-ec4f-4702-a189-d671c3db8c19 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.273563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24961337-ec4f-4702-a189-d671c3db8c19 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.273734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24961337-ec4f-4702-a189-d671c3db8c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.305998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-24961337-ec4f-4702-a189-d671c3db8c19 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:41.307135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-24961337-ec4f-4702-a189-d671c3db8c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:41.353556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd2596-0792-432d-8f3c-56665bab987a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.354168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd2596-0792-432d-8f3c-56665bab987a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.354640 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd2596-0792-432d-8f3c-56665bab987a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.356432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 230/457] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:41.361809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba4800-9f41-4f91-98b8-ccaf192cc9bd None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.362348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba4800-9f41-4f91-98b8-ccaf192cc9bd None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.363443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba4800-9f41-4f91-98b8-ccaf192cc9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.397866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 228/458] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:41.398859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e5ba4800-9f41-4f91-98b8-ccaf192cc9bd None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:41.400082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e5ba4800-9f41-4f91-98b8-ccaf192cc9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:41.434124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24961337-ec4f-4702-a189-d671c3db8c19 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:41.487543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:41.518794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 231/459] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:41.555615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba4800-9f41-4f91-98b8-ccaf192cc9bd None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:41.556692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:41.568124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24961337-ec4f-4702-a189-d671c3db8c19 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:41.624626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74af5c-0745-4d7f-a108-88fdd3a1fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:41.625258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74af5c-0745-4d7f-a108-88fdd3a1fbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.625533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74af5c-0745-4d7f-a108-88fdd3a1fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.625797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74af5c-0745-4d7f-a108-88fdd3a1fbf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.645659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74af5c-0745-4d7f-a108-88fdd3a1fbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:41.647116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74af5c-0745-4d7f-a108-88fdd3a1fbf0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-fb74af5c-0745-4d7f-a108-88fdd3a1fbf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:41.663853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:41.664904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:41.665771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:41.665771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:41.665771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:41.667845 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 229/460] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:41.669298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfaee5-9605-4b1a-8687-123c4fc54633 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.669784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfaee5-9605-4b1a-8687-123c4fc54633 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.670256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfaee5-9605-4b1a-8687-123c4fc54633 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.671973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 230/461] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:41.673443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfaee5-9605-4b1a-8687-123c4fc54633 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:41.723141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba4800-9f41-4f91-98b8-ccaf192cc9bd None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:41.768055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b41f8-5665-4230-b32c-a58c02926572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:41.769611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b41f8-5665-4230-b32c-a58c02926572 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.770057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b41f8-5665-4230-b32c-a58c02926572 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.770336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b41f8-5665-4230-b32c-a58c02926572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.788605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b41f8-5665-4230-b32c-a58c02926572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:41.790671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b41f8-5665-4230-b32c-a58c02926572 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:41.810280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfaee5-9605-4b1a-8687-123c4fc54633 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-dd6b41f8-5665-4230-b32c-a58c02926572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:41.814922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:41.815923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:41.817446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:41.817446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:41.817446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:41.820497 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 232/462] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:41.822045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47991c3-ed41-4f4f-88ba-d82257878f9d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.824736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47991c3-ed41-4f4f-88ba-d82257878f9d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.825046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47991c3-ed41-4f4f-88ba-d82257878f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.850336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c47991c3-ed41-4f4f-88ba-d82257878f9d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:41.851559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c47991c3-ed41-4f4f-88ba-d82257878f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:41.853373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:41.854089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:41.854507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:41.854792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:41.855446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:41.855748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:41.857270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:41.876117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 231/463] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:41.877035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:41.880042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:41.886934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:41.945230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 233/464] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:41.946383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47991c3-ed41-4f4f-88ba-d82257878f9d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:42.016268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a61120-3de2-4223-a03c-604d32bf6085 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:42.065327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:42.066910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.067643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.068142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.068990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:42.069509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:42.071233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:42.076363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47991c3-ed41-4f4f-88ba-d82257878f9d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:42.090457 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 232/465] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:42.091761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:42.112871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098fdbf-f465-4925-9ca0-216a7593146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:42.113539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098fdbf-f465-4925-9ca0-216a7593146e None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.113824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098fdbf-f465-4925-9ca0-216a7593146e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.114179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098fdbf-f465-4925-9ca0-216a7593146e None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.115100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098fdbf-f465-4925-9ca0-216a7593146e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:42.116735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098fdbf-f465-4925-9ca0-216a7593146e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:42.148247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 234/466] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:41 2022] POST /identity/v3/users => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:42.149322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098fdbf-f465-4925-9ca0-216a7593146e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:42.170541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:42.199845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:42.219145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a7293-562a-4a51-ae09-f71ccb61e8e1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:42.260315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b5d67-ef10-494b-9786-e9728a0ecae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:42.261319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b5d67-ef10-494b-9786-e9728a0ecae9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.261826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b5d67-ef10-494b-9786-e9728a0ecae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.262368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b5d67-ef10-494b-9786-e9728a0ecae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.274908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b5d67-ef10-494b-9786-e9728a0ecae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:42.277283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b5d67-ef10-494b-9786-e9728a0ecae9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:42.277892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098fdbf-f465-4925-9ca0-216a7593146e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9a0b5d67-ef10-494b-9786-e9728a0ecae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:42.295177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:42.296223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:42.297079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:42.297079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:42.297079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:42.299181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 233/467] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:42.301327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ed12-6bf0-45a2-a19e-fe3f6324b18a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.301929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ed12-6bf0-45a2-a19e-fe3f6324b18a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.302400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5ed12-6bf0-45a2-a19e-fe3f6324b18a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.304362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 234/468] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:42.306281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ed12-6bf0-45a2-a19e-fe3f6324b18a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:42.329683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ee261-1c87-47bb-854d-2d0165a340bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:42.330221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ee261-1c87-47bb-854d-2d0165a340bc None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.330495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ee261-1c87-47bb-854d-2d0165a340bc None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.330762 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ee261-1c87-47bb-854d-2d0165a340bc None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.331428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ee261-1c87-47bb-854d-2d0165a340bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:42.333546 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ee261-1c87-47bb-854d-2d0165a340bc None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:42.357769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 235/469] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:42 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:42.363721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b206-c544-4bdf-aeda-64e264bbe07f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.363721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b206-c544-4bdf-aeda-64e264bbe07f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.363721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069b206-c544-4bdf-aeda-64e264bbe07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.388196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3069b206-c544-4bdf-aeda-64e264bbe07f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:42.389562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3069b206-c544-4bdf-aeda-64e264bbe07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:42.453266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5ed12-6bf0-45a2-a19e-fe3f6324b18a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:42.470075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:42.472594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 236/470] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:42.494445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069b206-c544-4bdf-aeda-64e264bbe07f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:42.496753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:42.497620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.498117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.498621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.499564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:42.500110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:42.501758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:42.515873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 235/471] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:42.519104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:42.522425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:42.615131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069b206-c544-4bdf-aeda-64e264bbe07f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:42.650506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa2f6f-8c14-4ead-9f73-1bbfde9a58ef None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:42.661736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:42.662760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.663317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.663736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.676281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:42.678140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:42.688818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b8ad3-1eb4-4c09-b3ae-6003029d97b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:42.689662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8ad3-1eb4-4c09-b3ae-6003029d97b3 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.690023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8ad3-1eb4-4c09-b3ae-6003029d97b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.690306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b8ad3-1eb4-4c09-b3ae-6003029d97b3 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.691138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b8ad3-1eb4-4c09-b3ae-6003029d97b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:42.692715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b8ad3-1eb4-4c09-b3ae-6003029d97b3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:42.696377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:42.697478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:42.698599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:42.698599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:42.698599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:42.700956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 237/472] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:42.711322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:42.726017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 236/473] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:42 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:42.783360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:42.817416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa52b1-c604-49c2-b3bb-4c30f6e7c0df None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.818319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa52b1-c604-49c2-b3bb-4c30f6e7c0df None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.818789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa52b1-c604-49c2-b3bb-4c30f6e7c0df None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.821131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 237/474] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:42.827278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da836-6383-4c21-8513-d8898941edc0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.828118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da836-6383-4c21-8513-d8898941edc0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.829052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082da836-6383-4c21-8513-d8898941edc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.851211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-082da836-6383-4c21-8513-d8898941edc0 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:42.852814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-082da836-6383-4c21-8513-d8898941edc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:42.862783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4a69d-42a7-41be-b9d7-3ff56909f78c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:42.910217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:42.942315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 238/475] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:42.974260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082da836-6383-4c21-8513-d8898941edc0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:42.974940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:42.975574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:42.975856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:42.976218 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:42.976885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:42.977190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:42.978568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:43.003234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 238/476] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:43.008492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:43.118986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082da836-6383-4c21-8513-d8898941edc0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:43.139920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18550e4-2528-4d1a-a3dd-ad17c9406bb3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:43.162341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:43.170037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:43.171095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:43.171527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:43.172225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:43.173496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:43.175696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:43.186227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.sql.core [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 16:15:43.186227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 11 16:15:43.186227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [parameters: {'id': 'c3f35e319ba9481e86c6d0a4d1b095fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 11 16:15:43.186227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 16:15:43.186793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1100ac2-d90d-443b-8290-dff1750bae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:43.187575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100ac2-d90d-443b-8290-dff1750bae2f None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:43.188040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100ac2-d90d-443b-8290-dff1750bae2f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:43.188725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1100ac2-d90d-443b-8290-dff1750bae2f None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:43.189478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1100ac2-d90d-443b-8290-dff1750bae2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:43.191186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1100ac2-d90d-443b-8290-dff1750bae2f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 11 16:15:43.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return ref.to_dict() Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 16:15:43.210297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application yield resource Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise self.value Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application session.commit() Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 16:15:43.211553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.flush() Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application compat.raise_( Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 16:15:43.212859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 16:15:43.214901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 16:15:43.216634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [parameters: {'id': 'c3f35e319ba9481e86c6d0a4d1b095fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:43.218314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 11 16:15:43.220162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:43.224497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 239/477] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Aug 11 16:15:42 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:43.229011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 239/478] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:43 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:43.230135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:43.282380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:43.403792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f38d64-14df-4826-8969-d27e6dc357df None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:43.451334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881e9fd-5d68-470c-a35f-5c7fe37b92a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:43.452315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e9fd-5d68-470c-a35f-5c7fe37b92a4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:43.452824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e9fd-5d68-470c-a35f-5c7fe37b92a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:43.453287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e9fd-5d68-470c-a35f-5c7fe37b92a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:43.465400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881e9fd-5d68-470c-a35f-5c7fe37b92a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d881e9fd-5d68-470c-a35f-5c7fe37b92a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:43.470835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:43.471747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 11 16:15:43.473165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:43.475363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 240/479] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:15:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:43.480883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1100ac2-d90d-443b-8290-dff1750bae2f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:43.597395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1100ac2-d90d-443b-8290-dff1750bae2f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:43.637049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e9931-d199-4f34-812d-0de408a4a7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:43.637687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e9931-d199-4f34-812d-0de408a4a7e1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:43.637968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e9931-d199-4f34-812d-0de408a4a7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:43.638244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e9931-d199-4f34-812d-0de408a4a7e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:43.638942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e9931-d199-4f34-812d-0de408a4a7e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:43.639247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e9931-d199-4f34-812d-0de408a4a7e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:43.640867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e9931-d199-4f34-812d-0de408a4a7e1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:43.651471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 240/480] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:44.595947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdccf9-9f7a-4b38-b606-9f16783b530e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:44.596491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdccf9-9f7a-4b38-b606-9f16783b530e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:44.596491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdccf9-9f7a-4b38-b606-9f16783b530e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:44.598649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 241/481] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:44.604131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958cac6-056d-4630-8fe0-4249b177c9fe None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:44.604547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958cac6-056d-4630-8fe0-4249b177c9fe None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:44.604718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958cac6-056d-4630-8fe0-4249b177c9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:44.628316 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5958cac6-056d-4630-8fe0-4249b177c9fe None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:44.629584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5958cac6-056d-4630-8fe0-4249b177c9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:44.666620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:44.716001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 241/482] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:44.755037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdccf9-9f7a-4b38-b606-9f16783b530e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:44.882734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:44.908675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdccf9-9f7a-4b38-b606-9f16783b530e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:44.964844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e90a3-c06d-4a19-b3d6-523a43602666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:44.965485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e90a3-c06d-4a19-b3d6-523a43602666 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:44.965765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e90a3-c06d-4a19-b3d6-523a43602666 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:44.966038 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e90a3-c06d-4a19-b3d6-523a43602666 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:44.984995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e90a3-c06d-4a19-b3d6-523a43602666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3a4e90a3-c06d-4a19-b3d6-523a43602666 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:44.990144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:44.991040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:44.991854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 242/483] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:44.997135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958cac6-056d-4630-8fe0-4249b177c9fe None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:45.031810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6bd2-ba48-4dd1-bcb2-963f0a844817 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.032144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6bd2-ba48-4dd1-bcb2-963f0a844817 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.032430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6bd2-ba48-4dd1-bcb2-963f0a844817 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.034005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 243/484] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:45.038575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61cb89-95b6-42b0-b83f-7a7467d440d8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.038930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61cb89-95b6-42b0-b83f-7a7467d440d8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.039203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61cb89-95b6-42b0-b83f-7a7467d440d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.073382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7b61cb89-95b6-42b0-b83f-7a7467d440d8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:45.074638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7b61cb89-95b6-42b0-b83f-7a7467d440d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:45.201633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958cac6-056d-4630-8fe0-4249b177c9fe None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:45.203073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 244/485] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:45.234601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61cb89-95b6-42b0-b83f-7a7467d440d8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:45.256784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:45.257405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.257685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.257958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.258633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:45.258940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:45.260338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:45.285578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 242/486] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:45.289583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:45.343875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:45.378646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:45.425995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61cb89-95b6-42b0-b83f-7a7467d440d8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:45.433187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3f21e-5375-457c-a5d0-356871975503 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:45.471026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:45.471597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.471882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.472807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.486043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:45.487570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:45.493844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520efbf-f5e7-41dd-86e4-83ba764e03cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:45.494496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520efbf-f5e7-41dd-86e4-83ba764e03cf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.494773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520efbf-f5e7-41dd-86e4-83ba764e03cf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.495039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520efbf-f5e7-41dd-86e4-83ba764e03cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:45.505115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:45.506044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:45.506908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:45.506908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:45.506908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:45.506908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 243/487] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:45.515074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520efbf-f5e7-41dd-86e4-83ba764e03cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:45.517056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6520efbf-f5e7-41dd-86e4-83ba764e03cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:45.524394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:45.526584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 245/488] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:45.528608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1da07-beb8-4f16-a73c-211c5619c451 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.530436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1da07-beb8-4f16-a73c-211c5619c451 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.530436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1da07-beb8-4f16-a73c-211c5619c451 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.530436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 246/489] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:45.536289 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1da07-beb8-4f16-a73c-211c5619c451 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:45.606377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:45.635050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af97317-53c2-42d2-a15d-2a57898807a3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:45.641529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1da07-beb8-4f16-a73c-211c5619c451 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:45.674043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:45.675095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.675761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.676204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.677045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665317df-ddeb-4250-be5b-6478bc7f11cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:45.677233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:45.677676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:45.677841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665317df-ddeb-4250-be5b-6478bc7f11cb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.678187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665317df-ddeb-4250-be5b-6478bc7f11cb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.678524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665317df-ddeb-4250-be5b-6478bc7f11cb None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.679253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665317df-ddeb-4250-be5b-6478bc7f11cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:45.679622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665317df-ddeb-4250-be5b-6478bc7f11cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:45.680009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:45.681140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665317df-ddeb-4250-be5b-6478bc7f11cb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:45.682542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:45.691010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 247/490] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:45.692722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb71009-1869-4f56-8efc-f6fe94b93782 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.693079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb71009-1869-4f56-8efc-f6fe94b93782 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.693464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb71009-1869-4f56-8efc-f6fe94b93782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.696404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 244/491] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:45.697397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:45.717477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4bb71009-1869-4f56-8efc-f6fe94b93782 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:45.718797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4bb71009-1869-4f56-8efc-f6fe94b93782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:45.790444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 248/492] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:45.791587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb71009-1869-4f56-8efc-f6fe94b93782 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:45.809364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af3ca-b4da-4f51-a0c6-84ce70bdf600 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:45.848615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:45.849484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.849983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.850424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.857884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:45.864162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:45.865891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:45.883935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:45.885080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:45.885940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:45.885940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:45.885940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:45.888036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 245/493] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:45.889030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:45.899919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb71009-1869-4f56-8efc-f6fe94b93782 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:45.929300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:45.939926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:45.940794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:45.941142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:45.941479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:45.949523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:45.951129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:45.964533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:45.965481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:15:45.966982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:45.970189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 249/494] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:15:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:45.971031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.004147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d532ec-b2c6-48ed-8db6-151d3ba892dd None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.043393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545694f3-c683-4c1f-9dbe-11c81f1df4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.043942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545694f3-c683-4c1f-9dbe-11c81f1df4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.044248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545694f3-c683-4c1f-9dbe-11c81f1df4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.044523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545694f3-c683-4c1f-9dbe-11c81f1df4e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.058242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545694f3-c683-4c1f-9dbe-11c81f1df4e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-545694f3-c683-4c1f-9dbe-11c81f1df4e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:46.065027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:46.066509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:46.067759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 246/495] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:46.068732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485b83-a4a1-4bea-ad2e-5b1661ac0099 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.068801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485b83-a4a1-4bea-ad2e-5b1661ac0099 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.069080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485b83-a4a1-4bea-ad2e-5b1661ac0099 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.070722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 247/496] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:46.071840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2485b83-a4a1-4bea-ad2e-5b1661ac0099 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.091498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f626b97-fcf3-49ea-8ca8-f03ea5a6b739 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.126759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:46.130522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.132279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.132734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.133173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.134131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:46.134746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:46.136206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:46.149156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 250/497] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:46.149956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.184583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2485b83-a4a1-4bea-ad2e-5b1661ac0099 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.200522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:46.221420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f1369-30c2-4c60-8af6-20e1d96e7a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.222173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f1369-30c2-4c60-8af6-20e1d96e7a7f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.222640 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f1369-30c2-4c60-8af6-20e1d96e7a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.223055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f1369-30c2-4c60-8af6-20e1d96e7a7f None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.223906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f1369-30c2-4c60-8af6-20e1d96e7a7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:46.224270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f1369-30c2-4c60-8af6-20e1d96e7a7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:46.225677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f1369-30c2-4c60-8af6-20e1d96e7a7f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:46.258983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c7525-efd8-41d4-bda3-4561a857fff1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.267859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 248/498] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:46.267859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9208e88-bcf9-4cbf-9cba-3f6c7f6a8f6a None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.267859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9208e88-bcf9-4cbf-9cba-3f6c7f6a8f6a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.267859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9208e88-bcf9-4cbf-9cba-3f6c7f6a8f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.285310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f9208e88-bcf9-4cbf-9cba-3f6c7f6a8f6a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:46.286520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f9208e88-bcf9-4cbf-9cba-3f6c7f6a8f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:46.298012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.298545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.298829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.299117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.299769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:46.300691 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:46.302116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:46.315512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 251/499] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:46.315512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.372187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 249/500] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:46.372187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9208e88-bcf9-4cbf-9cba-3f6c7f6a8f6a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.419632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:46.429401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903cb40-b0c9-4da5-8a33-73423f1298b6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.438071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:46.463666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.464328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.464608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.464883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.472922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:46.474387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:46.479713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9208e88-bcf9-4cbf-9cba-3f6c7f6a8f6a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:46.485195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:46.486119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 252/501] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:46.487027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.515084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002e2f2-24ec-4aa9-9bff-5cbf7412a280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.515685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002e2f2-24ec-4aa9-9bff-5cbf7412a280 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.515957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002e2f2-24ec-4aa9-9bff-5cbf7412a280 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.516255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002e2f2-24ec-4aa9-9bff-5cbf7412a280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.583634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002e2f2-24ec-4aa9-9bff-5cbf7412a280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:46.584674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c167d7-dde5-460e-bb56-6fcd5f469f14 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.585151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002e2f2-24ec-4aa9-9bff-5cbf7412a280 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:46.616640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.617258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.617533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.617799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.623159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:46.630176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:46.631657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:46.645878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:46.647216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:46.648470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:46.648470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:46.648470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:46.648470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 253/502] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:46.648470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.669187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:46.689101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 250/503] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:46.690367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f2a52-6924-427b-9367-d55ef3ce38cf None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.691176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f2a52-6924-427b-9367-d55ef3ce38cf None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.691448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f2a52-6924-427b-9367-d55ef3ce38cf None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.692739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 251/504] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:46.693427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f2a52-6924-427b-9367-d55ef3ce38cf None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.749641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e1bab-e848-4f84-93cb-7bdcf26cbc99 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.785159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9a118-2d1d-4c03-8025-4dc76a3bccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.785784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9a118-2d1d-4c03-8025-4dc76a3bccce None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.786065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9a118-2d1d-4c03-8025-4dc76a3bccce None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.786337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9a118-2d1d-4c03-8025-4dc76a3bccce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.794425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f2a52-6924-427b-9367-d55ef3ce38cf None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.796418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9a118-2d1d-4c03-8025-4dc76a3bccce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-36d9a118-2d1d-4c03-8025-4dc76a3bccce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:46.801589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:46.802531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:46.803330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 254/505] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:46.804003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9a118-2d1d-4c03-8025-4dc76a3bccce None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.834505 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.835128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.835472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.835810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.836633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:46.836937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:46.838386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:46.846493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:46.856451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 252/506] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:46.857237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:46.886598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:46.905816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9a118-2d1d-4c03-8025-4dc76a3bccce None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.938756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905bfc9-dddc-4165-bd59-a823ff0a7744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.939353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905bfc9-dddc-4165-bd59-a823ff0a7744 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.939634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905bfc9-dddc-4165-bd59-a823ff0a7744 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.939901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905bfc9-dddc-4165-bd59-a823ff0a7744 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.940568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905bfc9-dddc-4165-bd59-a823ff0a7744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:46.942057 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905bfc9-dddc-4165-bd59-a823ff0a7744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:46.943361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905bfc9-dddc-4165-bd59-a823ff0a7744 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:46.954774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 255/507] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:46.955900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b213785-b4a6-444a-ac25-6226580535ba None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.956244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b213785-b4a6-444a-ac25-6226580535ba None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.956662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b213785-b4a6-444a-ac25-6226580535ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:46.957081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998d781-2a52-4fa7-8b76-9ba0440543fa None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:46.973228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3b213785-b4a6-444a-ac25-6226580535ba None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:46.974400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3b213785-b4a6-444a-ac25-6226580535ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:46.992538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:46.993158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:46.993436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:46.993708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.004138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.005586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:47.020267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:47.021208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:47.022139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:47.022139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:47.022139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:47.022139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 253/508] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:47.022524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.047284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 256/509] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:47.048199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b213785-b4a6-444a-ac25-6226580535ba None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.079505 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.118105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.130729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8608cec5-5d08-400c-afda-f09ef0c87a72 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.148884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b213785-b4a6-444a-ac25-6226580535ba None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.163531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.164183 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.164470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.164745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.165414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:47.165717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.167089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.174833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 254/510] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:47.176466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.183679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449d2d-5f53-4be2-ae5b-9cfd86db8ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.184310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449d2d-5f53-4be2-ae5b-9cfd86db8ccf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.184588 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449d2d-5f53-4be2-ae5b-9cfd86db8ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.184855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51449d2d-5f53-4be2-ae5b-9cfd86db8ccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.252402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51449d2d-5f53-4be2-ae5b-9cfd86db8ccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.254535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51449d2d-5f53-4be2-ae5b-9cfd86db8ccf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.277678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec81ec-48e3-4151-b2c0-b0cc117e75a9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.288360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.311581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.312233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.312508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.312777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.320329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.321779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.321892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:47.333893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:47.334822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 255/511] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:47.335726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.356266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 257/512] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:47.357103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449d2d-5f53-4be2-ae5b-9cfd86db8ccf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.449273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452085-30d7-42eb-ac49-35d26f769a3f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.455088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51449d2d-5f53-4be2-ae5b-9cfd86db8ccf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.483903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.484526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.484804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.485072 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.485713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:47.486006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.487375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.488620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.489234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.489506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.489770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.499212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 256/513] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:47.499985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.501135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.502571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:47.517014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:47.518122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:47.519397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:47.519397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:47.519397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:47.519397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 258/514] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:47.519397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.537150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.542810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.606721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b58b0-1a90-476f-a7e1-d593deebd3d1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.617830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c73af-5e9a-44eb-b50a-9b9ca32469a6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.640708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.641327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.641608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.641878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.650965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.651556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.651832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.652125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.652220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.652849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:47.653147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.653571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.654543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:47.668271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:47.669341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:47.670422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:47.670422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:47.670422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:47.672249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 257/515] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:47.673258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.690980 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 259/516] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:47.691798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.761075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.776219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.779694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed7d32-ffa1-45f0-8a69-de5b68c7c721 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.795199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15a1b7-066e-4b66-8e55-1cbbe051b19a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.812584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.813458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.813738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.814012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.824692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.826144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.829006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.829632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.829913 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.830187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.836699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.838118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:47.839795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:47.840802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:47.841664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:47.841664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:47.841664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:47.841664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 258/517] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:47.841664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:47.849960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:47.850796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:47.851654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 260/518] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:47.852660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.944647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc0ca-80a4-4f7a-9471-d20ec6fae770 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.952843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7ea33-b12c-4893-ada8-7ce8ea48ca34 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:47.976445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.977034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.977307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.977581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.978207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:47.978504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.979849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.981236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:47.985282 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:47.985885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:47.986168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:47.986440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:47.987084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:47.987386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:47.989328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:47.992356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 259/519] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:47.993142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:47.999295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.003212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 261/520] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:48.005000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.098139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c96e46-f1ea-471f-b626-8abfe23aeb9c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.103948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3156b-ac64-448c-beac-c23d3d99a7e7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.132793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e7f67-b5b1-48ed-bb5a-5e67b62e6db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.133406 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e7f67-b5b1-48ed-bb5a-5e67b62e6db1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.133687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e7f67-b5b1-48ed-bb5a-5e67b62e6db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.133959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e7f67-b5b1-48ed-bb5a-5e67b62e6db1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.138285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.138876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.139150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.139416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.140061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:48.140389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.142069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.152905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 262/521] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:48.153683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.197854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.205981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e7f67-b5b1-48ed-bb5a-5e67b62e6db1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.207646 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e7f67-b5b1-48ed-bb5a-5e67b62e6db1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.212445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.254244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993ef5b-81e9-4e17-8b07-b9b103e2ca41 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.286764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.287367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.287646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.287917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.288642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:48.289229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.290663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.305954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 260/522] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:48.306689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e7f67-b5b1-48ed-bb5a-5e67b62e6db1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.313815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 263/523] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:48.313815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.399039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.406481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e7f67-b5b1-48ed-bb5a-5e67b62e6db1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.411202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebb7f5-e313-4e7c-b2dd-ad8a2ff03551 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.430711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.441870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78f5db-d4fa-489d-aebe-09a955f8ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.442467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78f5db-d4fa-489d-aebe-09a955f8ac55 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.442746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78f5db-d4fa-489d-aebe-09a955f8ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.443016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78f5db-d4fa-489d-aebe-09a955f8ac55 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.443654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78f5db-d4fa-489d-aebe-09a955f8ac55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.444990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.445140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78f5db-d4fa-489d-aebe-09a955f8ac55 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.445621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.445901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.446175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.453137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.454574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.465300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 261/524] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:15:48 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:15:48.466425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78f5db-d4fa-489d-aebe-09a955f8ac55 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:48.466878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:48.467915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 264/525] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:48.468883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.584859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f379fef-24da-499c-9c99-987ef2a08211 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.591394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78f5db-d4fa-489d-aebe-09a955f8ac55 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.624644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.625218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.625499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.625777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.628406 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.629046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.629327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.629603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.637387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.637759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.639228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.639503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:48.653948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:48.655303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:48.656284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:48.656284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:48.656284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:48.656284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:48.656284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:48.656284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:48.656284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:48.656284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:48.656702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:48.658638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:48.660877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:48.660877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:48.660877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:48.661057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 262/526] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:48.661057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.661180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 265/527] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:48.661180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.676418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.700887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.774842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ccda-94f7-42ad-bc39-e680227598cf None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.778176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea12cce-1850-44f9-9789-30c5c1672300 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.809673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.810284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.810599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.810834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.811565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=2e3819e78802469c83c09a1cdd865f58) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:48.811876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.813621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.814175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.814775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.815050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.815497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.817261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:48.817584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.819001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:48.828423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 263/528] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/role_assignments?user.id=2e3819e78802469c83c09a1cdd865f58&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:48.828423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.852916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 266/529] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:48.853737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:48.921991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.951038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3c56e-0e4f-4d76-8165-e8303feb6ae9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.977381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:48.988818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c3350-bf59-4ba8-853b-b6374d1f459f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:48.993799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:48.994029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:48.994282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:48.994550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:48.995247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:48.995548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:48.998138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:49.023256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 264/530] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:49.024219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:49.038276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:49.038896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:49.039171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:49.039440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:49.040562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:49.040882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:49.042245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:49.059459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 267/531] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:49.059459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:49.149968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2bd54-6e0d-4945-b6b4-abd745e72e98 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:49.162079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85e76a-3c9a-4186-a61c-6d33c9b91545 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:49.192288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272f4a6-0c09-4824-8bd7-143fc1421967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:49.193279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272f4a6-0c09-4824-8bd7-143fc1421967 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:49.193812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272f4a6-0c09-4824-8bd7-143fc1421967 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:49.194386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272f4a6-0c09-4824-8bd7-143fc1421967 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:49.197613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:49.204313 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:49.205221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:49.205702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:49.206733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:49.207909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=f7a2237c18934cb2aa3699a5776874ad) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:49.209001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:49.211451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:49.219604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 268/532] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:49 2022] GET /identity/v3/role_assignments?user.id=f7a2237c18934cb2aa3699a5776874ad&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:49.220682 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:49.265712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:49.332547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272f4a6-0c09-4824-8bd7-143fc1421967 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:49.334389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272f4a6-0c09-4824-8bd7-143fc1421967 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:49.419729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300a9f1-72cd-4fa9-b2ca-2682ad949116 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:49.465052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:49.466062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:49.466646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:49.467121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:49.475146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:49.476931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:49.492498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 265/533] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:49.499745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272f4a6-0c09-4824-8bd7-143fc1421967 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:49.501151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:49.502029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:49.502890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:49.502890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:49.502890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:49.502890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:49.502890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:49.502890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:49.502890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:49.502890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:49.510147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 269/534] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:49.512149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:49.546682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:49.606318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:49.643045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272f4a6-0c09-4824-8bd7-143fc1421967 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:49.660154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6a6da-8602-4262-9dbc-39bf49357244 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:49.687144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:49.689847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:49.691506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:49.693306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:49.718342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:49.720224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:49.747231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:49.748449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:49.749477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:49.749477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:49.749477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:49.749477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 266/535] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:49.751869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:49.752614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:49.753286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:49.753645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:49.754654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:49.755051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:49.756637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:49.759153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:49.780566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 270/536] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:49.786971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:49.869444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:49.887851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc466e-6531-483d-824f-baced0aab4f8 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:49.931120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:49.931776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:49.932051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:49.932342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:49.932979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:49.933275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:49.934621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:49.954141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 267/537] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:49.961124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:49.980828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059f2e-68d9-46ca-ab2c-c37b79b11403 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:50.038339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:50.041853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:50.041853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:50.041853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:50.041853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=89757c5dc48942259ffdd67e0cd19ab0) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:50.042126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:50.043474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:50.048420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:50.062458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 271/538] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:49 2022] GET /identity/v3/role_assignments?user.id=89757c5dc48942259ffdd67e0cd19ab0&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:50.130625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668406b8-30f7-4d04-a4ba-6d113d0a4e37 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:50.176757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:50.177831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:50.178317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:50.178786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:50.192273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:50.194133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:50.212085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:50.212995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:50.213865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:50.213865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:50.213865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:50.213865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:50.213865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:50.213865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:50.213865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:50.213865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:50.216486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 268/539] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:50.223077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:50.375418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b3223-9dc9-42c6-8e0d-b57528c2fbb4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:50.411222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2af402-07f8-4876-804c-17819bbbea86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:50.411832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2af402-07f8-4876-804c-17819bbbea86 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:50.412130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2af402-07f8-4876-804c-17819bbbea86 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:50.412403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2af402-07f8-4876-804c-17819bbbea86 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:50.413097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2af402-07f8-4876-804c-17819bbbea86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:50.413399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2af402-07f8-4876-804c-17819bbbea86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:50.414815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2af402-07f8-4876-804c-17819bbbea86 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:50.429373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 272/540] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:50.435451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:50.450846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:50.592683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b77165-f6ad-4bb2-a6c8-67c3e73f792e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:50.641882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1543b33-e076-4633-8547-bf4f1ef4376f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:50.642868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1543b33-e076-4633-8547-bf4f1ef4376f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:50.643357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1543b33-e076-4633-8547-bf4f1ef4376f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:50.644258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1543b33-e076-4633-8547-bf4f1ef4376f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:50.645240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1543b33-e076-4633-8547-bf4f1ef4376f None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=123d33bb6caa464aba4f9be6b0fb7656) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:50.645748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1543b33-e076-4633-8547-bf4f1ef4376f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:50.647402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1543b33-e076-4633-8547-bf4f1ef4376f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:50.656810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 269/541] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:50 2022] GET /identity/v3/role_assignments?user.id=123d33bb6caa464aba4f9be6b0fb7656&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:50.889869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c265-336c-4dce-b64b-2a273d116541 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:50.890346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c265-336c-4dce-b64b-2a273d116541 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:50.890346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c265-336c-4dce-b64b-2a273d116541 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:50.891955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 273/542] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:50.897992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca0d60-5045-4064-8b36-a47892a8b1db None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:50.898399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca0d60-5045-4064-8b36-a47892a8b1db None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:50.898526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca0d60-5045-4064-8b36-a47892a8b1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:50.922135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-9cca0d60-5045-4064-8b36-a47892a8b1db None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:50.923349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9cca0d60-5045-4064-8b36-a47892a8b1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:51.018139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:51.024243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 270/543] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:51.070805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0c265-336c-4dce-b64b-2a273d116541 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:51.147571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:51.204975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0c265-336c-4dce-b64b-2a273d116541 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:51.252818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce35c03-89d9-43d8-a494-40a7a5013798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:51.253839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce35c03-89d9-43d8-a494-40a7a5013798 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.254276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce35c03-89d9-43d8-a494-40a7a5013798 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.254622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce35c03-89d9-43d8-a494-40a7a5013798 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.266800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce35c03-89d9-43d8-a494-40a7a5013798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-fce35c03-89d9-43d8-a494-40a7a5013798 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:51.272261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:51.273148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:51.273961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:51.278534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 274/544] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:51.279902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca0d60-5045-4064-8b36-a47892a8b1db None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:51.394624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca0d60-5045-4064-8b36-a47892a8b1db None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:51.408805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3cc1-db4b-4134-9896-d2bdfbccaf60 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.409070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3cc1-db4b-4134-9896-d2bdfbccaf60 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.409972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3cc1-db4b-4134-9896-d2bdfbccaf60 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.411743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 275/545] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:15:51.421248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f242e-daf7-443d-b319-2dd0f51b07e5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.421530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f242e-daf7-443d-b319-2dd0f51b07e5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.421800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f242e-daf7-443d-b319-2dd0f51b07e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.445923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cb6f242e-daf7-443d-b319-2dd0f51b07e5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:51.447122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cb6f242e-daf7-443d-b319-2dd0f51b07e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:51.465212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:51.465895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.466177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.466452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.467130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:51.467434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:51.468825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:51.490017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 271/546] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:51.492247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:51.555242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 276/547] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:51.581978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f242e-daf7-443d-b319-2dd0f51b07e5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:51.617425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:51.668230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:51.678850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6817be-fe30-4947-8ab6-2fdb1aad2f65 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:51.717088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71166263-5518-400e-b052-0fd949d3f8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:51.718031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71166263-5518-400e-b052-0fd949d3f8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.718451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71166263-5518-400e-b052-0fd949d3f8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.718893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71166263-5518-400e-b052-0fd949d3f8b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.731646 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71166263-5518-400e-b052-0fd949d3f8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:51.731848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f242e-daf7-443d-b319-2dd0f51b07e5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:51.733578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71166263-5518-400e-b052-0fd949d3f8b2 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-71166263-5518-400e-b052-0fd949d3f8b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:51.749228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:51.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:51.751352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:51.751352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:51.751352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:51.753308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 272/548] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:51.754986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ff08-6a17-4459-8ed4-e309e8838adf None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.755449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ff08-6a17-4459-8ed4-e309e8838adf None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.755907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ff08-6a17-4459-8ed4-e309e8838adf None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.757648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 273/549] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:51.758614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ff08-6a17-4459-8ed4-e309e8838adf None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:51.775936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa57291-b14f-4ccb-906f-88d73661f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:51.776901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa57291-b14f-4ccb-906f-88d73661f230 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.777183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa57291-b14f-4ccb-906f-88d73661f230 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.777456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa57291-b14f-4ccb-906f-88d73661f230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.791729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa57291-b14f-4ccb-906f-88d73661f230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-cfa57291-b14f-4ccb-906f-88d73661f230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:51.798130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:51.799053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:51.801136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 277/550] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:51.802552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ba41c-56fd-4ac7-a709-544d0c7fa1cd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.802552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ba41c-56fd-4ac7-a709-544d0c7fa1cd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.802552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ba41c-56fd-4ac7-a709-544d0c7fa1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.824602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-162ba41c-56fd-4ac7-a709-544d0c7fa1cd None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:51.825840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-162ba41c-56fd-4ac7-a709-544d0c7fa1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:51.881413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ff08-6a17-4459-8ed4-e309e8838adf None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:51.897995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:51.916788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 278/551] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:51.919141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ba41c-56fd-4ac7-a709-544d0c7fa1cd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:51.933389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:51.934413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:51.934894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:51.935380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:51.936302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:51.936810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:51.939707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:51.953775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 274/552] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:51.954703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:51.981213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:52.066201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ba41c-56fd-4ac7-a709-544d0c7fa1cd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.082794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe047b4-7e70-4609-adef-c8ed1d0641a5 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.104503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.105498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.106028 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.106478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.107312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:52.107854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.109516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.123551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 279/553] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:52.124675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:52.125221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a69e87-2349-42e6-8630-d7ae778e98e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.125928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a69e87-2349-42e6-8630-d7ae778e98e9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.126279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a69e87-2349-42e6-8630-d7ae778e98e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.126621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a69e87-2349-42e6-8630-d7ae778e98e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.141683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a69e87-2349-42e6-8630-d7ae778e98e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-b5a69e87-2349-42e6-8630-d7ae778e98e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:52.147061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:52.148062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:52.149019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:52.153055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 275/554] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:52.153884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a69e87-2349-42e6-8630-d7ae778e98e9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:52.190154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:52.248040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:52.252208 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01318f1c-7592-4733-8a09-0d8505fc23be None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.261139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a69e87-2349-42e6-8630-d7ae778e98e9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.296785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.297707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.298010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.298204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.299030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.299504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.300488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.301094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.312158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.312369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.314110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.314362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:52.326783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:52.327815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:15:52.328792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:52.333174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:52.334125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:52.335501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:52.335501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:52.335501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:52.335678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 280/555] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:52.335678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:52.337617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 276/556] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:52.345566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:52.436552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:52.447664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388a0fc-2d4e-4812-b543-de171ef41974 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.453944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616459d-6bcd-445e-a243-27dda19d4263 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.483673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.484366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.484637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.484909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.485632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:52.486009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.488092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.491047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.491936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.492454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.492928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.493764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:52.494257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.495795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.501330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 281/557] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:52.501330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:52.513365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 277/558] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:52.514545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:52.525362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:52.605717 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e17d9-17e0-41bb-a12f-71d7ee955f31 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.622436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f802d5-17e8-43d8-910e-6a239cdf1f94 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.641633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da4d24-d93b-4763-8a10-0272976f49d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.642241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da4d24-d93b-4763-8a10-0272976f49d0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.642520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da4d24-d93b-4763-8a10-0272976f49d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.642793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da4d24-d93b-4763-8a10-0272976f49d0 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.643528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da4d24-d93b-4763-8a10-0272976f49d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:52.644435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da4d24-d93b-4763-8a10-0272976f49d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.646209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da4d24-d93b-4763-8a10-0272976f49d0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.658861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.659604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.659875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.660173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.668789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 282/559] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:52.668789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb105f-35a0-4017-9b3f-2e65613ab48a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.668789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb105f-35a0-4017-9b3f-2e65613ab48a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.668789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb105f-35a0-4017-9b3f-2e65613ab48a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.668789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 283/560] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:52.668789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb105f-35a0-4017-9b3f-2e65613ab48a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:52.675006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.676479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.683000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:52.692701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:52.693678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:52.694578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:52.694578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:52.694578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:52.696010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 278/561] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:52.696938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:52.768155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:52.789649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb105f-35a0-4017-9b3f-2e65613ab48a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.808973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a2ca-6712-44c4-b849-4da51a221cf7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:52.827612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baf1ee-a691-40a5-866b-e34ba7d22c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.828272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf1ee-a691-40a5-866b-e34ba7d22c27 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.828548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf1ee-a691-40a5-866b-e34ba7d22c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.828818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf1ee-a691-40a5-866b-e34ba7d22c27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.837181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baf1ee-a691-40a5-866b-e34ba7d22c27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.838604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baf1ee-a691-40a5-866b-e34ba7d22c27 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.844049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526b801-5a8e-4cbc-9101-cd221031a813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:52.844712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526b801-5a8e-4cbc-9101-cd221031a813 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.844983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526b801-5a8e-4cbc-9101-cd221031a813 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.845257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526b801-5a8e-4cbc-9101-cd221031a813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b4baf1ee-a691-40a5-866b-e34ba7d22c27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:52.850542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:52.851530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:52.852545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 284/562] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:52.853150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df558-f2d7-4e03-8749-ea6f6b9b5451 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:52.853150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df558-f2d7-4e03-8749-ea6f6b9b5451 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:52.853283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49df558-f2d7-4e03-8749-ea6f6b9b5451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:52.873841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e49df558-f2d7-4e03-8749-ea6f6b9b5451 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:52.875027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e49df558-f2d7-4e03-8749-ea6f6b9b5451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:52.920754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:52.926718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526b801-5a8e-4cbc-9101-cd221031a813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:52.928210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526b801-5a8e-4cbc-9101-cd221031a813 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:52.944352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 285/563] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:52.945256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49df558-f2d7-4e03-8749-ea6f6b9b5451 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.012900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.028973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 279/564] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:53.029591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526b801-5a8e-4cbc-9101-cd221031a813 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.045929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49df558-f2d7-4e03-8749-ea6f6b9b5451 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.080041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992a541-9f2d-400d-a406-00c583c0d0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.080678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a541-9f2d-400d-a406-00c583c0d0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.080958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a541-9f2d-400d-a406-00c583c0d0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.081735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992a541-9f2d-400d-a406-00c583c0d0d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.082460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992a541-9f2d-400d-a406-00c583c0d0d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:53.082761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992a541-9f2d-400d-a406-00c583c0d0d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.084136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992a541-9f2d-400d-a406-00c583c0d0d3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.094991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 286/565] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:53.096224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014348e-8a4c-4acf-a9cb-3ca873a0676e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.096504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014348e-8a4c-4acf-a9cb-3ca873a0676e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.097484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014348e-8a4c-4acf-a9cb-3ca873a0676e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.102888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 287/566] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:53.102888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014348e-8a4c-4acf-a9cb-3ca873a0676e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.133845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.137069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526b801-5a8e-4cbc-9101-cd221031a813 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.172454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.173055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.173330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.173603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.174345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:53.174625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.176129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.193014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 280/567] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:53.193795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.206620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014348e-8a4c-4acf-a9cb-3ca873a0676e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.235414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.242579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693ef3a-0d0e-4e53-bae3-6400ce5ddf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.243199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693ef3a-0d0e-4e53-bae3-6400ce5ddf4f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.243474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693ef3a-0d0e-4e53-bae3-6400ce5ddf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.243741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693ef3a-0d0e-4e53-bae3-6400ce5ddf4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.254128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e693ef3a-0d0e-4e53-bae3-6400ce5ddf4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e693ef3a-0d0e-4e53-bae3-6400ce5ddf4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:53.258782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:53.259703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 288/568] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:53.261199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693ef3a-0d0e-4e53-bae3-6400ce5ddf4f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.298406 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0674b81-9857-46bb-8708-bf8fb36226df None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.333184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf14ba4-b680-4dc3-82b3-81159533e950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.333800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf14ba4-b680-4dc3-82b3-81159533e950 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.334080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf14ba4-b680-4dc3-82b3-81159533e950 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.334355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf14ba4-b680-4dc3-82b3-81159533e950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.344509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf14ba4-b680-4dc3-82b3-81159533e950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.345940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf14ba4-b680-4dc3-82b3-81159533e950 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.358941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693ef3a-0d0e-4e53-bae3-6400ce5ddf4f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-0bf14ba4-b680-4dc3-82b3-81159533e950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:53.360755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:53.361755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:53.363258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:53.363258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:53.363258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:53.363258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 281/569] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:53.363879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0bff-b751-4b3c-9610-fb950f35bc5b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.363879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0bff-b751-4b3c-9610-fb950f35bc5b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.364150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0bff-b751-4b3c-9610-fb950f35bc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.385680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f28a0bff-b751-4b3c-9610-fb950f35bc5b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:53.386877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f28a0bff-b751-4b3c-9610-fb950f35bc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:53.388874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.395100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.395728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.396002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.396298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.403321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.404785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:53.417480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:53.418514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 289/570] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:53.420013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.459967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 282/571] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:53.464985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a0bff-b751-4b3c-9610-fb950f35bc5b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.468887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.520799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d25ef-1907-4421-a1e1-e0be808a57c4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.555914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.556573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.556998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.557271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.567602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a0bff-b751-4b3c-9610-fb950f35bc5b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.569535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:53.583470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:53.584504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:53.585464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:53.585464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:53.585464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:53.589180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 290/572] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:53.589880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.603188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.603792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.604068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.604368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.605029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:53.605327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.606691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.611584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.616862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 283/573] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:53.617734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.690093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93455e28-2a11-4d91-be22-7102e24e0bf9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.691613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.718126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce58aa-6c13-4ec1-8e87-e9849292dfbf None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.724530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.725128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.725410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.725673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.726303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:53.726600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.727946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.740026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 291/574] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:53.740836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.752321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.752955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.753233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.753500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.754409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:53.754708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.756126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.773820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 284/575] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:53.774598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.832478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.842968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc486683-184d-43d6-8969-bf1377708e81 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.873457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d07ae-cbac-4cd3-a60f-7cf3fc7b4066 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:53.876802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e71989-d698-4345-94df-95de28e05dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.877432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e71989-d698-4345-94df-95de28e05dbd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.877713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e71989-d698-4345-94df-95de28e05dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.878013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e71989-d698-4345-94df-95de28e05dbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.888006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e71989-d698-4345-94df-95de28e05dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-95e71989-d698-4345-94df-95de28e05dbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:53.892670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:53.894327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 292/576] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:53.900180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e71989-d698-4345-94df-95de28e05dbd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.910099 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:53.910712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:53.910991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:53.911259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:53.911901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:53.912231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:53.913582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:53.922905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:53.924936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 285/577] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:53.925733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:53.999335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e71989-d698-4345-94df-95de28e05dbd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.026961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dbb88-c4d2-411f-8067-bd6de88d4af1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.032841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.033536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.033820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.034095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.044310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.044629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.045793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:54.062032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:54.063158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:54.064695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:54.064695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:54.064695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:54.064886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.064886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.064886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.065097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 293/578] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:54.065167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.065235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.072557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.074002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:54.085878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:54.086741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 286/579] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:54.087926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.140777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.165885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16fa60-5d80-4d04-a68b-059433285299 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.188087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75193cd-d82a-4e9c-b6e6-f231d2207e37 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.198741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.199360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.199633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.199900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.210599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.212048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.222344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.222935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.223212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.223478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.224164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:54.224463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.225803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:54.226005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:54.227031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:54.227929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:54.227929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:54.227929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:54.227929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 294/580] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:54.227929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.237717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 287/581] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:54.238615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.280986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.333510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3219d4c-5109-46e9-a545-5ed457bc9ab2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.338794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d62449-3d50-4aa7-9e55-6f92cbf181d3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.361171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.370141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.370765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.371040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.371308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.373413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.374014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.374291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.374564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.375210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:54.375512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.376916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.377716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.379163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.388673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 288/582] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:54.389466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:54.389929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:54.390813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 295/583] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:54.397190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.488057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.489518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417a239-94e6-4a53-a0a8-d204711f9d3c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.491199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57012a-31da-4017-bbfd-bfcd852db43e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.523932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.524173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.524565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.524670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.524844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.524946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.525113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.525213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.525750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:54.525848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:54.526051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.526168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.527408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.527507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.539137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 289/584] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:54.543961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 296/585] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:54.544137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.544225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.590311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.643394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74577110-f453-4a3b-8c18-e7642a64c5cf None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.646156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c96ccd-8f7f-4071-a7c0-9fac047efa32 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.676850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.677462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.677737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.678000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.680257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.680876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.681151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.681424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.688189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.688549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.689869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.690095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:54.701662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:54.702601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 297/586] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:54.703549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:54.704805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:54.705687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:54.707470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:54.707470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:54.707470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:54.707470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 290/587] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:54.707470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.721833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.806454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b06dcd-7244-49b9-8b45-97a20bda2694 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.806675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63328343-5991-461a-83a4-0ffcaab020c1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.814192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.840226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.840893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.841130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.841400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.842686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.843287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.843562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.843832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.844546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:54.844841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.846256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.848702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.850126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.858640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 298/588] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:54.859473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:54.861208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:54.862078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 291/589] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:54.864736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:54.935837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:54.961355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b43f1a-db63-4e5d-8b33-ea3054e4f659 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.965606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c06fcf-dfd4-4042-a7e3-f23f8888e6d1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:54.995166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.995766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.996047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:54.996350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:54.997075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=256cc27b87684392a22f36429b960644) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:54.997382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:54.998842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:54.998959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:54.999543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:54.999818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.000084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.000955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:55.001256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.002658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.004104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 299/590] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/role_assignments?user.id=256cc27b87684392a22f36429b960644&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:55.004894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.031095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 292/591] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Aug 11 16:15:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:55.031095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.060047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:55.142273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216ce31-91a8-4d45-bc82-531116951598 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.148833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114744c-bd8f-4df2-8cbb-732c1b3efc54 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.178825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:55.184286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:55.184975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:55.185310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.185585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.185792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82d4e3-071e-471d-b29e-7f806141e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:55.186230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:55.186453 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82d4e3-071e-471d-b29e-7f806141e64e None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:55.186551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.186733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82d4e3-071e-471d-b29e-7f806141e64e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.187002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82d4e3-071e-471d-b29e-7f806141e64e None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/2e3819e78802469c83c09a1cdd865f58/roles/7bd0567f3a3c43628a4ec259a52f1e24` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.187875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.200356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 300/592] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:55.201188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.208933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82d4e3-071e-471d-b29e-7f806141e64e None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=2e3819e78802469c83c09a1cdd865f58, role_id=7bd0567f3a3c43628a4ec259a52f1e24)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.210388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82d4e3-071e-471d-b29e-7f806141e64e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.225018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 293/593] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:55 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/2e3819e78802469c83c09a1cdd865f58/roles/7bd0567f3a3c43628a4ec259a52f1e24 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:55.225667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82d4e3-071e-471d-b29e-7f806141e64e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.323206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:55.335775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf4407-709d-4140-bdb0-7de70abe000c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.379024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:55.380330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:55.381251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.381990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.383190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:55.383686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.386049 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.390168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82d4e3-071e-471d-b29e-7f806141e64e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.403952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 301/594] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:55.405079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.436867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:55.437502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:55.437783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.438055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.457427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.459615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:55.478508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:55.479514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:55.480473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:55.480473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:55.480473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:55.480473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 294/595] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:55.481233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.516583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:55.529272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e3b3b-fddf-4a1a-a3d0-bfac9ed54733 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.567424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:55.568210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:55.568598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.568931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.577254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.578848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:55.592499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:55.593499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:55.594489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 302/596] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:55.595088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.610122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:55.613774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bef68b-640a-48ff-84af-b14b77e39df8 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.660946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6620d0-e263-448a-9599-0cff28df7f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:55.661853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6620d0-e263-448a-9599-0cff28df7f29 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:55.662272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6620d0-e263-448a-9599-0cff28df7f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.662679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6620d0-e263-448a-9599-0cff28df7f29 None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/f7a2237c18934cb2aa3699a5776874ad/roles/7bd0567f3a3c43628a4ec259a52f1e24` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.679580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6620d0-e263-448a-9599-0cff28df7f29 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=f7a2237c18934cb2aa3699a5776874ad, role_id=7bd0567f3a3c43628a4ec259a52f1e24)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.681080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6620d0-e263-448a-9599-0cff28df7f29 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.704209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 295/597] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:55 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/f7a2237c18934cb2aa3699a5776874ad/roles/7bd0567f3a3c43628a4ec259a52f1e24 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:55.705255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6620d0-e263-448a-9599-0cff28df7f29 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.735677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43185a9-4514-498f-b80a-285f5aa4226d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.784156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:55.786848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:55.786848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.786848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.790810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:55.790810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.790810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.809605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 303/598] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:55.818249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:55.886848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:55.915778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6620d0-e263-448a-9599-0cff28df7f29 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.941221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:55.967115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:55.968366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:55.968647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:55.968921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:55.969960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:55.970270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:55.973267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:55.981822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8939a-9f57-406b-a064-bf2164e6b018 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:55.998960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 296/599] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Aug 11 16:15:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:56.005163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:56.025626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:56.026315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:56.026544 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:56.026814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:56.033888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:56.035994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:56.051458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:56.052430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:56.054455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 304/600] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:56.058602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:56.172210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62771c-a097-4c24-989e-c3ab32e46904 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:56.200296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac55a3-b8b3-4d41-be90-336dc6df9728 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:56.232551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:56.240392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:56.241162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:56.241440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:56.241560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:56.241745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:56.242095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:56.242376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:56.242485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:56.242652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:56.242758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:56.244316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:56.259515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:56.265416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 305/601] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:15:56.269346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:56.274102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:56.275468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:56.305214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:56.306435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:56.307548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:56.307548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:56.307548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:56.316147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 297/602] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:56.320364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:56.406731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5a734-f632-48d0-8948-b6b9b5bc07a9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:56.451678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70ef43-cb85-4f0c-8ef1-711b09ed5067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:56.452602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ef43-cb85-4f0c-8ef1-711b09ed5067 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:56.453065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ef43-cb85-4f0c-8ef1-711b09ed5067 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:56.453490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ef43-cb85-4f0c-8ef1-711b09ed5067 None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/89757c5dc48942259ffdd67e0cd19ab0/roles/7bd0567f3a3c43628a4ec259a52f1e24` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:56.476333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbceef6b-42bb-4498-8785-c15480d2e043 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:56.482129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70ef43-cb85-4f0c-8ef1-711b09ed5067 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=89757c5dc48942259ffdd67e0cd19ab0, role_id=7bd0567f3a3c43628a4ec259a52f1e24)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:56.483767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70ef43-cb85-4f0c-8ef1-711b09ed5067 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:56.512294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 306/603] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:56 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/89757c5dc48942259ffdd67e0cd19ab0/roles/7bd0567f3a3c43628a4ec259a52f1e24 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:56.524391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:56.525088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:56.525375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:56.525652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:56.526322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:56.526632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:56.528090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:56.543884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 298/604] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:56.553269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70ef43-cb85-4f0c-8ef1-711b09ed5067 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:56.627220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:56.713812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70ef43-cb85-4f0c-8ef1-711b09ed5067 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:56.763945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:56.765010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:56.765497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:56.765994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:56.775902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:56.777661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:15:56.795171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:56.796166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:56.797105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:15:56.797105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:15:56.797105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:15:56.797105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:15:56.797105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:15:56.797105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:15:56.797105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:15:56.797105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:56.799764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 307/605] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:56.805832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:56.875374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:56.932705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b2530-7397-49d8-beed-fabc5e02a796 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:56.974332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d253c06-9e0b-4eb3-8f2d-69af16fc1718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:56.974923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253c06-9e0b-4eb3-8f2d-69af16fc1718 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:56.975132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253c06-9e0b-4eb3-8f2d-69af16fc1718 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:56.979186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d253c06-9e0b-4eb3-8f2d-69af16fc1718 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:56.979186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d253c06-9e0b-4eb3-8f2d-69af16fc1718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:56.979186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d253c06-9e0b-4eb3-8f2d-69af16fc1718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:56.979186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d253c06-9e0b-4eb3-8f2d-69af16fc1718 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:56.990566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 299/606] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:56.996134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:57.125362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:57.129316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14660dbe-9a7f-4fef-8f33-fdbb54e6c645 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:57.171568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc51e-7bd6-4218-88c0-36f03e97a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:57.172260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc51e-7bd6-4218-88c0-36f03e97a43a None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:57.172542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc51e-7bd6-4218-88c0-36f03e97a43a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:57.172817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc51e-7bd6-4218-88c0-36f03e97a43a None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/123d33bb6caa464aba4f9be6b0fb7656/roles/7bd0567f3a3c43628a4ec259a52f1e24` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:57.198541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fc51e-7bd6-4218-88c0-36f03e97a43a None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=123d33bb6caa464aba4f9be6b0fb7656, role_id=7bd0567f3a3c43628a4ec259a52f1e24)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:57.200022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fc51e-7bd6-4218-88c0-36f03e97a43a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:57.212561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2842f-b9c1-4cd1-bfb8-ebe148df1297 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:57.212872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2842f-b9c1-4cd1-bfb8-ebe148df1297 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:57.213179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2842f-b9c1-4cd1-bfb8-ebe148df1297 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:57.214961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 300/607] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:57.215990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 308/608] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:15:56 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/123d33bb6caa464aba4f9be6b0fb7656/roles/7bd0567f3a3c43628a4ec259a52f1e24 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:15:57.220859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520113e9-a9f0-4f31-95ef-74735e062392 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:57.221146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520113e9-a9f0-4f31-95ef-74735e062392 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:57.221401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520113e9-a9f0-4f31-95ef-74735e062392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:57.254981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-520113e9-a9f0-4f31-95ef-74735e062392 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:57.256669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-520113e9-a9f0-4f31-95ef-74735e062392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:57.372281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 301/609] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:57.401645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc51e-7bd6-4218-88c0-36f03e97a43a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:57.520488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc51e-7bd6-4218-88c0-36f03e97a43a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:57.562234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be7be8-00a0-48c2-8945-93152edfa580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:57.562880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be7be8-00a0-48c2-8945-93152edfa580 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:57.563161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be7be8-00a0-48c2-8945-93152edfa580 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:57.563697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be7be8-00a0-48c2-8945-93152edfa580 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:57.583674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be7be8-00a0-48c2-8945-93152edfa580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-44be7be8-00a0-48c2-8945-93152edfa580 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:57.593034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:57.594527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:57.596508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 309/610] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:57.606843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520113e9-a9f0-4f31-95ef-74735e062392 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:57.670063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:57.745888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520113e9-a9f0-4f31-95ef-74735e062392 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:57.799048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74d6cf-ffdd-4ea5-8f80-2c1a8e47af17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:57.799586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74d6cf-ffdd-4ea5-8f80-2c1a8e47af17 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:57.800823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74d6cf-ffdd-4ea5-8f80-2c1a8e47af17 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:57.801235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74d6cf-ffdd-4ea5-8f80-2c1a8e47af17 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:57.801923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74d6cf-ffdd-4ea5-8f80-2c1a8e47af17 None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:57.802280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74d6cf-ffdd-4ea5-8f80-2c1a8e47af17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:57.803675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74d6cf-ffdd-4ea5-8f80-2c1a8e47af17 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:57.819337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 302/611] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:57.825231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be7be8-00a0-48c2-8945-93152edfa580 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:57.828321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c858-6965-4153-9d9c-59e1449aac94 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:57.828617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c858-6965-4153-9d9c-59e1449aac94 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:57.828877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664c858-6965-4153-9d9c-59e1449aac94 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:57.832171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 303/612] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:57.844792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5da7-6f59-4a4c-b4f5-993218cebc39 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:57.845075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5da7-6f59-4a4c-b4f5-993218cebc39 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:57.845348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5da7-6f59-4a4c-b4f5-993218cebc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:57.848674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:57.876256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-47bc5da7-6f59-4a4c-b4f5-993218cebc39 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:57.878687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-47bc5da7-6f59-4a4c-b4f5-993218cebc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:57.952116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be7be8-00a0-48c2-8945-93152edfa580 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:57.999830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:58.000748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.001196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.001632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.022562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:58.025375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:58.041008 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 304/613] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:58.052226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:58.054244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:58.055526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:58.055526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:58.055526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:58.055526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 310/614] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:58.066000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5da7-6f59-4a4c-b4f5-993218cebc39 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.075314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.156205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:58.160246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:58.209536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f272588-112d-45f7-852e-78fa122a9e93 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:58.251360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56176d60-ec71-47d3-a886-c5cbdd43d6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:58.252445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176d60-ec71-47d3-a886-c5cbdd43d6ec None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.252893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176d60-ec71-47d3-a886-c5cbdd43d6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.253690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56176d60-ec71-47d3-a886-c5cbdd43d6ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.269524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56176d60-ec71-47d3-a886-c5cbdd43d6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-56176d60-ec71-47d3-a886-c5cbdd43d6ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:58.278015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:58.279053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:58.279911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 311/615] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:15:58.282457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5da7-6f59-4a4c-b4f5-993218cebc39 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:58.285974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a90d12-7a02-4d10-bb47-afea6d250cc3 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.286263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a90d12-7a02-4d10-bb47-afea6d250cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.286538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a90d12-7a02-4d10-bb47-afea6d250cc3 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.291977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 312/616] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:58.291977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a90d12-7a02-4d10-bb47-afea6d250cc3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.330677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a0731-9633-4c33-964c-006dd2632a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:58.331932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a0731-9633-4c33-964c-006dd2632a91 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.332507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a0731-9633-4c33-964c-006dd2632a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.333008 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a0731-9633-4c33-964c-006dd2632a91 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.333816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a0731-9633-4c33-964c-006dd2632a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:58.334265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a0731-9633-4c33-964c-006dd2632a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:58.335789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a0731-9633-4c33-964c-006dd2632a91 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:58.348834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 305/617] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:58.350337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38729554-8f38-4438-8e24-ec7fff048164 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.350789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38729554-8f38-4438-8e24-ec7fff048164 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.351208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38729554-8f38-4438-8e24-ec7fff048164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.370847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-38729554-8f38-4438-8e24-ec7fff048164 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:58.372431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-38729554-8f38-4438-8e24-ec7fff048164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:58.407392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a90d12-7a02-4d10-bb47-afea6d250cc3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:58.433025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:58.447511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:58.448146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.448424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.448749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.449421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:58.449723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:58.451107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:58.456510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 306/618] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:58.457708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38729554-8f38-4438-8e24-ec7fff048164 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.464256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 313/619] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:58.472147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.500790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:58.588930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a04e60-5bb3-482e-9950-0ef8ad6a7c8f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:58.606814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38729554-8f38-4438-8e24-ec7fff048164 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:58.627327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:58.628662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.629447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.629826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.641333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:58.642915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:58.654555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:58.655378 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.655804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.656255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:58.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:58.661319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:58.662443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:58.662443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:58.662443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:58.667449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 314/620] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:58.667449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.673591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:58.675603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:58.693344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:58.694690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:15:58.695879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:58.698752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 307/621] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:58.700009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.710016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:58.793226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1432ab-8ed8-4aa7-9dfe-e5689c3ac37b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:58.804362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:58.819259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7d4e5-cd8d-439b-bbfb-ef20ec3d1f74 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:58.844123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12b80f-96a4-4b15-acf1-f697f6663607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:58.844925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b80f-96a4-4b15-acf1-f697f6663607 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.845317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b80f-96a4-4b15-acf1-f697f6663607 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.845685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b80f-96a4-4b15-acf1-f697f6663607 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.860528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12b80f-96a4-4b15-acf1-f697f6663607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:15:58.866050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6f12b80f-96a4-4b15-acf1-f697f6663607 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:15:58.866335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:58.867241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:15:58.868130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:58.870003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:58.870486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:58.870961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:58.871818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:58.872301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 315/622] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:58.872596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:58.873145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12b80f-96a4-4b15-acf1-f697f6663607 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.874551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:58.895664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 308/623] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:58.895664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:58.983246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:58.991094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12b80f-96a4-4b15-acf1-f697f6663607 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.037089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2a32d-0492-492e-98dc-c67a4d997d41 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.037918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.041138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.044478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.044862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.046036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:59.046213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.047753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.074456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.074989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 316/624] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:59.075730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.075932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.076359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.076812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.077651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:59.078126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.079683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.089528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 309/625] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:15:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:59.090593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.102221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:59.198329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8a869-ee2a-4470-8e40-78b4735de2a9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.201606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b472d-2956-4dca-81ff-5ef2b905940e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.234897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.235882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.236393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.236866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.241149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.241786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.242066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.242340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.249965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.250151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.251609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.252147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:15:59.265472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:59.266479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:15:59.267423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:59.271145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:59.272085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:59.273352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:59.273352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:59.273352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:15:59.273526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 317/626] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:59.273597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.275361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 310/627] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:59.276389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.292090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:59.339970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:59.384634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97304d-e1d1-4bb3-acf9-78ae6a18dee6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.389215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297e956-4e74-40a6-a590-67cb396bb7d9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.425040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.426091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.426371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.426636 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.427341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b906078-5050-40c1-b7a4-334a16cc811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.427475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:59.427653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.428282 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b906078-5050-40c1-b7a4-334a16cc811f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.428655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b906078-5050-40c1-b7a4-334a16cc811f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.429019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b906078-5050-40c1-b7a4-334a16cc811f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.429118 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.439995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b906078-5050-40c1-b7a4-334a16cc811f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.442350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b906078-5050-40c1-b7a4-334a16cc811f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.449627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 311/628] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:59.450287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1b906078-5050-40c1-b7a4-334a16cc811f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:15:59.462134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:15:59.463130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:15:59.464048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:15:59.464048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:15:59.464048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:15:59.464048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 318/629] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:15:59.465008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e252-b0d4-4e68-8ecb-4f0482a17705 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.465008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e252-b0d4-4e68-8ecb-4f0482a17705 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.465008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1e252-b0d4-4e68-8ecb-4f0482a17705 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.466327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 319/630] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:59.466860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1e252-b0d4-4e68-8ecb-4f0482a17705 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.537377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:59.567093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fbbea-acf0-41a9-807b-a96e6fd8bb41 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.576187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1e252-b0d4-4e68-8ecb-4f0482a17705 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.584808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:59.606477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.607081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.607357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.607617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.608301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:59.608603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.609957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.613764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deac3a-65ad-4962-bd62-8ae52f4c5099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.614607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deac3a-65ad-4962-bd62-8ae52f4c5099 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.614979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deac3a-65ad-4962-bd62-8ae52f4c5099 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.615148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deac3a-65ad-4962-bd62-8ae52f4c5099 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.628398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 312/631] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:59.628398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.693278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deac3a-65ad-4962-bd62-8ae52f4c5099 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.694650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deac3a-65ad-4962-bd62-8ae52f4c5099 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.732238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe10f0f-fbc4-43df-8f19-894a03ac8e79 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:15:59.758277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:59.765856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9c383-ec3c-41c6-8e9e-cfbba94fbea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:15:59.766464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9c383-ec3c-41c6-8e9e-cfbba94fbea7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.766742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9c383-ec3c-41c6-8e9e-cfbba94fbea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.767013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9c383-ec3c-41c6-8e9e-cfbba94fbea7 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.767665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9c383-ec3c-41c6-8e9e-cfbba94fbea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:15:59.767968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9c383-ec3c-41c6-8e9e-cfbba94fbea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:15:59.770069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9c383-ec3c-41c6-8e9e-cfbba94fbea7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:15:59.782920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 313/632] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:15:59.784962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26310d-51aa-4437-8bd0-0d1a497b5cac None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.785339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26310d-51aa-4437-8bd0-0d1a497b5cac None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.785500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26310d-51aa-4437-8bd0-0d1a497b5cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.804651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-6d26310d-51aa-4437-8bd0-0d1a497b5cac None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:15:59.806004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6d26310d-51aa-4437-8bd0-0d1a497b5cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:15:59.838500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:15:59.880259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 314/633] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:15:59.881111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26310d-51aa-4437-8bd0-0d1a497b5cac None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.911911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 320/634] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:15:59.916390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802150e-5d04-43d9-b07c-8e5dd375c509 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:15:59.916390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802150e-5d04-43d9-b07c-8e5dd375c509 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:15:59.916390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802150e-5d04-43d9-b07c-8e5dd375c509 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:15:59.916390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 321/635] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:15:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:15:59.916390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802150e-5d04-43d9-b07c-8e5dd375c509 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:15:59.995718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26310d-51aa-4437-8bd0-0d1a497b5cac None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.016131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802150e-5d04-43d9-b07c-8e5dd375c509 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.030412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.031028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.031304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.031570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.038184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.039651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:00.050733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:00.051666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:00.056364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:00.056364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:00.056364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:00.056364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:00.056364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:00.056364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:00.056364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:00.056364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:00.057323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.057323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.057323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.057554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 315/636] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:00.057624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.057693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.064886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.066864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:00.078076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:00.079063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 322/637] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:15:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:00.080017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.088705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:00.107628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:00.160773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ad65e-3055-408e-8eb6-55793233b70e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.178924 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5faeb-41be-4158-914e-d5634abd3448 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.194251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d7ba5-bff5-4d86-8860-464b81dae5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.194870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d7ba5-bff5-4d86-8860-464b81dae5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.195150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d7ba5-bff5-4d86-8860-464b81dae5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.195414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d7ba5-bff5-4d86-8860-464b81dae5e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.205197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d7ba5-bff5-4d86-8860-464b81dae5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-340d7ba5-bff5-4d86-8860-464b81dae5e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:00.210886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:00.211845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:00.213018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:00.213786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.214221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 316/638] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:00.214314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.214314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.214314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.218198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5414-2675-4dd9-be80-3ca89a6ea9ac None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.218484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5414-2675-4dd9-be80-3ca89a6ea9ac None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.218757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b5414-2675-4dd9-be80-3ca89a6ea9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.223109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.224552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.236473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c51b5414-2675-4dd9-be80-3ca89a6ea9ac None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:00.237917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c51b5414-2675-4dd9-be80-3ca89a6ea9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:00.238626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:00.239787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:00.240765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:00.240765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:00.240765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:00.240765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 323/639] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:00.241006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.308950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 317/640] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:00.309594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b5414-2675-4dd9-be80-3ca89a6ea9ac None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.319810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:00.337459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:00.344848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda8ec9-0e12-4107-9cdc-85af9d1ab219 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.379857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.380563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.380837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.381101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.381781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:00.382076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.383479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.395380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 324/641] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:00.396174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.427835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b5414-2675-4dd9-be80-3ca89a6ea9ac None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.461913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.462510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.462780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.463047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.463761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:00.464055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.465500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.483399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 318/642] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:00.484213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.505744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71942a04-16aa-448c-bef5-f040594a2648 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.537953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.538560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.538824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.539086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.539731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:00.540022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.541379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.546214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:00.555464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 325/643] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:00.556294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.557531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:00.583580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4b9f2-3c9a-42ce-9907-940c28c0d8bd None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.618297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.618891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.619178 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.619450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.620087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:00.620419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.621805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.633091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 319/644] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:00.633861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.657005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78050082-5deb-4633-8d1c-36840440a3a7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.691121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d1142-14b6-49d0-8c6e-328ed8861e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.691739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d1142-14b6-49d0-8c6e-328ed8861e2a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.692018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d1142-14b6-49d0-8c6e-328ed8861e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.692318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d1142-14b6-49d0-8c6e-328ed8861e2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.702056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d1142-14b6-49d0-8c6e-328ed8861e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8f2d1142-14b6-49d0-8c6e-328ed8861e2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:00.706728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:00.707868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 326/645] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:00.708959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d1142-14b6-49d0-8c6e-328ed8861e2a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.734239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c82f4b-ea44-4e18-b631-9003895f4128 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.768038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c0d7b-23f3-47a5-846d-9b5aa534b772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.769176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c0d7b-23f3-47a5-846d-9b5aa534b772 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.769586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c0d7b-23f3-47a5-846d-9b5aa534b772 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.770036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:00.770296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c0d7b-23f3-47a5-846d-9b5aa534b772 None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/256cc27b87684392a22f36429b960644/roles/7bd0567f3a3c43628a4ec259a52f1e24` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.772877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:00.787664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c0d7b-23f3-47a5-846d-9b5aa534b772 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=256cc27b87684392a22f36429b960644, role_id=7bd0567f3a3c43628a4ec259a52f1e24)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.789276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c0d7b-23f3-47a5-846d-9b5aa534b772 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.801642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 320/646] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:16:00 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/256cc27b87684392a22f36429b960644/roles/7bd0567f3a3c43628a4ec259a52f1e24 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:16:00.802464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c0d7b-23f3-47a5-846d-9b5aa534b772 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:00.826367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d1142-14b6-49d0-8c6e-328ed8861e2a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.864574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1e041-b6e8-47a5-9a92-4110d5a4ebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.865087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1e041-b6e8-47a5-9a92-4110d5a4ebfb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.865361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1e041-b6e8-47a5-9a92-4110d5a4ebfb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.865626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1e041-b6e8-47a5-9a92-4110d5a4ebfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.922086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c0d7b-23f3-47a5-846d-9b5aa534b772 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:00.943251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1e041-b6e8-47a5-9a92-4110d5a4ebfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.945099 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1e041-b6e8-47a5-9a92-4110d5a4ebfb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.964989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:00.965782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:00.966194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:00.966569 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:00.978505 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:00.980088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:00.996861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:00.998125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:00.999357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:00.999357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:00.999357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:00.999357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 321/647] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:00.999357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.017002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:01.037491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:01.071197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 327/648] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:16:01.072117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1e041-b6e8-47a5-9a92-4110d5a4ebfb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.119615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab931de7-2518-417f-bfb3-16002ac2279f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.157660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.158288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.158563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.159737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.166851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.168582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.176368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1e041-b6e8-47a5-9a92-4110d5a4ebfb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:01.180699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:01.181605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:01.182503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:01.182503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:01.182503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:01.182503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:01.182503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:01.182503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:01.182503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:01.182503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:01.185231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 322/649] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:01.185530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.216213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.216959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.217157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.217407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.218068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:01.218362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.219706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.228818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 328/650] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:01.233806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.259177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:01.287512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:01.295061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc6375-7cca-456b-9b7b-8c4e9c2a26d3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.332647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.333254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.333533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.333805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.334446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:01.334748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.336126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.352229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 323/651] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:01.352414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bb151-483b-4fa9-9bbe-79b8ad1eb7a1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.352513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.388994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.389549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.389828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.390098 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.407615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.409188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:01.424881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:01.425895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:01.426857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:01.426857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:01.426857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:01.431390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 329/652] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:01.432319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.458179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996f1f4-a097-4b6b-b458-6d6e3ab70ee1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.493637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.494247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.494522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.494790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.495505 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:01.495783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.497230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.510419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 324/653] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:01.511210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.525520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:01.530425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:01.575117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c11192-baa6-4cf8-94ae-5f1b31d45469 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.620470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.621431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.621623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b153a6-4ed3-428e-9fa8-b54e680eb816 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.622085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.622533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.637898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.640129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.657439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.658069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.658350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.658625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:01.660920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:01.661893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:01.662802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:01.662802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:01.662802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:01.669295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 330/654] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:01.669862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.670944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.672435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:01.684730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:01.685960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 325/655] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:01.686928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.773042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:01.801933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b1663-17a9-4203-aef0-c61610137ca4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.802993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67a025-6681-4ef6-a53b-1a997ba0582e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:01.829982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:01.842614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.843229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.843513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.843791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.844483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:01.844788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.846164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.847933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:01.849093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:01.849093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:01.849437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:01.850392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=2e3819e78802469c83c09a1cdd865f58) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:01.850820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:01.852501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:01.859257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 331/656] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:01.860251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:01.864556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 326/657] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/role_assignments?user.id=2e3819e78802469c83c09a1cdd865f58&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:01.865522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.021023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740835b4-8d93-4f71-a2fe-25bef2159e1e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.074186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.075101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.075557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.076019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.077018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:02.077409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a1311-e5b0-494d-9ffd-90ff83fd92d0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.077732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.079320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.090356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:02.102044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 327/658] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:02.103125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.123678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.124326 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.124604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.124873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.125523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:02.125824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.127178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.141703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 332/659] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:02.142696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.197942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:02.234886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962cf3-2cdd-4cb7-a12c-ab1bf3f3125c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.272454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4988c71-522b-4062-90a9-573e946201f9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.284345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.285304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.285804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.286291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.296314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.298565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:02.314830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:02.316393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:02.317313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:02.317313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:02.317313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:02.317313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:02.317313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:02.317313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:02.317313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:02.317313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:02.318966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c752-f00b-49c6-a567-5d1d84cf12c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.319540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 328/660] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:02.320501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.324990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c752-f00b-49c6-a567-5d1d84cf12c9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.324990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c752-f00b-49c6-a567-5d1d84cf12c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.324990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90c752-f00b-49c6-a567-5d1d84cf12c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.401323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:02.401968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90c752-f00b-49c6-a567-5d1d84cf12c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.403669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90c752-f00b-49c6-a567-5d1d84cf12c9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.456881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:02.463840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f9d7d-dd55-4033-855c-edad42700e21 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.508930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.509846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.510125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.510396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.519100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 333/661] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:16:02.520045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c752-f00b-49c6-a567-5d1d84cf12c9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.525573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.527038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:02.543759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:02.544924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:02.545901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 329/662] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:02.547578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.640747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90c752-f00b-49c6-a567-5d1d84cf12c9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.669857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b24b0b-473f-4051-b26e-0b9495e4874d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.685337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.686261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.686541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.686813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.687660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:02.687968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.689023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:02.689805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.711374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 334/663] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:02.711374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.714950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.716408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.716920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.717416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.722176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:02.730907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.732772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:02.750892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:02.751915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:02.753476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:02.753476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:02.753476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:02.756173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 330/664] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:02.757223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.839975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbcac2-c171-4f0f-b7b7-28fe18235e2b None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.875596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.876480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.876935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.877360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.878829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:02.879226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.880676 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.895572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1da942-0e64-4ab5-b5dd-a3d6ea40add2 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:02.898405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 335/665] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:02.898719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.936544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd2f51-0d66-462f-9641-4a4828356f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:02.937517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2f51-0d66-462f-9641-4a4828356f24 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.938019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2f51-0d66-462f-9641-4a4828356f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.938743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2f51-0d66-462f-9641-4a4828356f24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.939682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2f51-0d66-462f-9641-4a4828356f24 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=f7a2237c18934cb2aa3699a5776874ad) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:02.940832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2f51-0d66-462f-9641-4a4828356f24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:02.942478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2f51-0d66-462f-9641-4a4828356f24 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:02.950087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 331/666] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/role_assignments?user.id=f7a2237c18934cb2aa3699a5776874ad&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:02.951451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94590f8-50a6-4edd-b432-6967ed91d07d None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:02.951938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94590f8-50a6-4edd-b432-6967ed91d07d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:02.952448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94590f8-50a6-4edd-b432-6967ed91d07d None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:02.953893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 332/667] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:02.954820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94590f8-50a6-4edd-b432-6967ed91d07d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:02.976788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:03.004508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:03.033074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1b0d5-95ab-428f-9bd0-01c63f469f9e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.083564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0d9fa-6d18-4175-8317-2ab520a4fca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.084380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d9fa-6d18-4175-8317-2ab520a4fca4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.084753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d9fa-6d18-4175-8317-2ab520a4fca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.085025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0d9fa-6d18-4175-8317-2ab520a4fca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.085703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d9fa-6d18-4175-8317-2ab520a4fca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:03.086002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d9fa-6d18-4175-8317-2ab520a4fca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:03.088207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0d9fa-6d18-4175-8317-2ab520a4fca4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:03.092480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94590f8-50a6-4edd-b432-6967ed91d07d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.112160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 336/668] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:03.112160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8f5aa-bf16-4aa2-99c0-6d2124c382ee None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.112160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8f5aa-bf16-4aa2-99c0-6d2124c382ee None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.112160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8f5aa-bf16-4aa2-99c0-6d2124c382ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.131293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bee8f5aa-bf16-4aa2-99c0-6d2124c382ee None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:03.132509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bee8f5aa-bf16-4aa2-99c0-6d2124c382ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:03.142985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa48d2c-231e-4e9e-83d8-4040f0fec7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.143670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa48d2c-231e-4e9e-83d8-4040f0fec7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.143991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa48d2c-231e-4e9e-83d8-4040f0fec7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.144284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa48d2c-231e-4e9e-83d8-4040f0fec7b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.212415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 337/669] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:03.213582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8f5aa-bf16-4aa2-99c0-6d2124c382ee None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:03.234877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa48d2c-231e-4e9e-83d8-4040f0fec7b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:03.237621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa48d2c-231e-4e9e-83d8-4040f0fec7b0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:03.275903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:03.278996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:03.336348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8f5aa-bf16-4aa2-99c0-6d2124c382ee None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.367908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 333/670] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:16:03.368886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa48d2c-231e-4e9e-83d8-4040f0fec7b0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:03.375177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.375812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.376085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.376379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.385379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:03.387365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:03.402568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:03.403491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:03.404448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 338/671] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:03.404983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:03.494157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa48d2c-231e-4e9e-83d8-4040f0fec7b0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.522179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12407c-f0d5-4cc3-8603-fa9177a6b541 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.531747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04c2f5-2bd2-4ddb-8281-8723cb82c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.532402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04c2f5-2bd2-4ddb-8281-8723cb82c8af None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.532678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04c2f5-2bd2-4ddb-8281-8723cb82c8af None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.532947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04c2f5-2bd2-4ddb-8281-8723cb82c8af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.535605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:03.540052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:03.547860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04c2f5-2bd2-4ddb-8281-8723cb82c8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-ab04c2f5-2bd2-4ddb-8281-8723cb82c8af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:03.555136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:03.556089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:03.557080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:03.564754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.565067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 334/672] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:03.565740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.566092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.566380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.566957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04c2f5-2bd2-4ddb-8281-8723cb82c8af None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:03.579180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:03.581471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:03.597589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:03.598618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:03.599872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:03.599872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:03.599872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:03.603677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 339/673] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:03.604632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:03.678276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04c2f5-2bd2-4ddb-8281-8723cb82c8af None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.711921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8bd25-2447-4a3d-9ece-1e21ee7ce259 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.718377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.719248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.719732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.720295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.721191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:03.724169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:03.726609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:03.741387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 335/674] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:03.742656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:03.757943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.758965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.759785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.760307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.761557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:03.762294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:03.764064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:03.779522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 340/675] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:03.779522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:03.810128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:03.823873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:03.874868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675e15-be01-4af8-8fae-66e77695e9e4 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.892739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f63dc-d35f-4898-86f0-0b09f5aeb28c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:03.913472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.914455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.915039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.915513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.916446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:03.917031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:03.918822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:03.930483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:03.930987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:03.931269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:03.931539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:03.932297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=89757c5dc48942259ffdd67e0cd19ab0) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:03.932605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:03.934008 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 336/676] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:03.935176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:03.936547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:03.944815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 341/677] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/role_assignments?user.id=89757c5dc48942259ffdd67e0cd19ab0&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:03.946067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.127770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24386632-6307-4ae7-aa4b-7fab81e6ab02 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:04.128690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:04.139493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51952f-9b3b-419c-9a68-6abfdb24fed1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:04.145643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:04.172030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42353d44-77bc-4870-8ea1-91bc59cc755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:04.173020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42353d44-77bc-4870-8ea1-91bc59cc755d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.173574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42353d44-77bc-4870-8ea1-91bc59cc755d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.173849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42353d44-77bc-4870-8ea1-91bc59cc755d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.181696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:04.181696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.181919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.182155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.187691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42353d44-77bc-4870-8ea1-91bc59cc755d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:04.189495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42353d44-77bc-4870-8ea1-91bc59cc755d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:04.194463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:04.195483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-42353d44-77bc-4870-8ea1-91bc59cc755d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:04.207143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:04.208191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:04.212705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:04.212705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:04.212705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:04.212705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 337/678] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:04.212705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6ace8-98be-4559-af70-b05b0971555d None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.212705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6ace8-98be-4559-af70-b05b0971555d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.212705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6ace8-98be-4559-af70-b05b0971555d None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:04.213124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:04.213925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:04.214820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:04.214820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:04.214820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:04.214820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:04.214820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:04.214820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:04.214820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:04.214820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:04.217227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 338/679] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:04.217376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 342/680] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:04.217540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdcb59-2403-45c6-b404-b1c383a0c170 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.217689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdcb59-2403-45c6-b404-b1c383a0c170 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.218511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdcb59-2403-45c6-b404-b1c383a0c170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.220006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.241973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ccbdcb59-2403-45c6-b404-b1c383a0c170 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:04.244073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ccbdcb59-2403-45c6-b404-b1c383a0c170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:04.324596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 339/681] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:04.325866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdcb59-2403-45c6-b404-b1c383a0c170 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.336206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc8fdc-d27c-46c1-96e9-79363204aac2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:04.388316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:04.388929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.389205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.389477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.390196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:04.390411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:04.391764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:04.408186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 343/682] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:04.408186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.424520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:04.438582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:04.446620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdcb59-2403-45c6-b404-b1c383a0c170 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:04.484006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:04.485031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.485547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.486043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.486938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:04.487618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:04.489301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:04.502165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 340/683] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:04.503924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.511648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf74004-e3fc-45cc-afb0-01b7bc03dde7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:04.547920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7ec3e-9425-4e71-b931-fab5f4b043e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:04.548587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7ec3e-9425-4e71-b931-fab5f4b043e1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.548867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7ec3e-9425-4e71-b931-fab5f4b043e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.549138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7ec3e-9425-4e71-b931-fab5f4b043e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.564786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7ec3e-9425-4e71-b931-fab5f4b043e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-fcc7ec3e-9425-4e71-b931-fab5f4b043e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:04.570707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:04.571711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:04.573064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:04.576328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 344/684] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:04.577166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7ec3e-9425-4e71-b931-fab5f4b043e1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.648370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9bf08-341a-4d71-b661-7afe685db247 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:04.659732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:04.687297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:04.687925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.688256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.688838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] PATH_INFO: `/v3/users/swift` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.702423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7ec3e-9425-4e71-b931-fab5f4b043e1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:04.744852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:04.745861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.746346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.746821 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.747807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=123d33bb6caa464aba4f9be6b0fb7656) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:04.748391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:04.750352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:04.758663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 345/685] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:04 2022] GET /identity/v3/role_assignments?user.id=123d33bb6caa464aba4f9be6b0fb7656&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:04.759950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.819041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:04.820584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:04.832998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:04.834030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:04.835623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 341/686] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:04.844208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.866547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:04.904879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa1668-80e5-4b34-93f9-038bf1552fb6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:04.958817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:04.960130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:04.960414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:04.960695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:04.961363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:04.961662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:04.967602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:04.974681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:04.988087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 346/687] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:04.995116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:04.999397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180f239-67b6-4b0a-bb20-ebf2eb491ace None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:05.044871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:05.045659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.046267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.046542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.047555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:05.047862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:05.049611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:05.076662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 342/688] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Aug 11 16:16:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:05.082201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:05.132228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410b3eb-0dcb-49d0-9e5e-ab1614a1158c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:05.168377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:05.172126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:05.173113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.173589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.174051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.187136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:05.189237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:05.196860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2c378-f946-4cbb-bf80-eac17de772dd None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:05.205126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:05.206675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:05.207599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:05.207599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:05.207599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:05.209763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 347/689] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:05.217530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:05.242594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d310a-3f26-4737-af45-23e0e0b5b7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:05.243575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d310a-3f26-4737-af45-23e0e0b5b7ec None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.243575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d310a-3f26-4737-af45-23e0e0b5b7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.243799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d310a-3f26-4737-af45-23e0e0b5b7ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.285387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:05.327297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d310a-3f26-4737-af45-23e0e0b5b7ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:05.329115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d310a-3f26-4737-af45-23e0e0b5b7ec None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:05.348239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcb47d-0a4c-4073-a8bd-ae35acd15b47 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:05.387629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371772f-0d75-4c3e-9106-ca81854599dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:05.388620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371772f-0d75-4c3e-9106-ca81854599dc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.389080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371772f-0d75-4c3e-9106-ca81854599dc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.389531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371772f-0d75-4c3e-9106-ca81854599dc None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.390416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371772f-0d75-4c3e-9106-ca81854599dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:05.390898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371772f-0d75-4c3e-9106-ca81854599dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:05.392482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371772f-0d75-4c3e-9106-ca81854599dc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:05.405760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 348/690] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:05.407367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad2106-08a8-47a8-b5ff-979e2b51f0e1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.407831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad2106-08a8-47a8-b5ff-979e2b51f0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.408311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad2106-08a8-47a8-b5ff-979e2b51f0e1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.414852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 349/691] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:05.415967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad2106-08a8-47a8-b5ff-979e2b51f0e1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:05.441229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:05.480272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 343/692] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:16:05.481883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713236-202e-4f28-a540-a485cddd060a None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.482348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713236-202e-4f28-a540-a485cddd060a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.482892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94713236-202e-4f28-a540-a485cddd060a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.506160 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-94713236-202e-4f28-a540-a485cddd060a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:05.507626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-94713236-202e-4f28-a540-a485cddd060a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:05.546876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad2106-08a8-47a8-b5ff-979e2b51f0e1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:05.560775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:05.598129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:05.598129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.598129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.598129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.598994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 344/693] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:05.600595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713236-202e-4f28-a540-a485cddd060a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:05.610778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:05.612730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:05.634172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:05.635393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:05.636445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:05.636999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 350/694] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:16:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:05.637446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:05.745139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94713236-202e-4f28-a540-a485cddd060a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:05.748622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:05.770325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eaaac-8ff9-425e-bab4-1a9d03778727 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:05.798428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:05.800284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.801185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.801665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.809032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:05.809674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:05.809954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:05.810229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:05.816545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:05.818787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:05.822612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:05.823846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:05.837521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:05.838513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:05.839417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:05.839417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:05.839417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:05.839598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:05.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:05.841490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:05.841490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:05.841490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:05.843895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 345/695] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:05.844415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 351/696] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:05.845190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:05.849747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:05.879384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:05.964081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14425a-6e09-429a-8169-4913e5892868 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:05.988777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593844b-4d05-4792-b0cc-e6fcb2916318 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:06.012347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.013547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.014414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.015160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.016329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:06.017263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.020008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.025522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.026134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.026462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.026737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.027478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:06.027789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.029258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.036715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 352/697] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:06.037817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.059288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 346/698] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:16:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:06.059476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.077015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:06.178096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:06.180784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f1da9-1382-4711-b7d9-d2484fe65dcb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:06.181196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba3ce6-5a74-44fb-a8c7-8c52dabc5e9e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:06.218999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.219607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.219881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.220253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.220809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:06.221108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.222664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.228740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.230100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.230607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.231099 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.239344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 353/699] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:06.239632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.240420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.241580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:06.256224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:06.257614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 347/700] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:06.258524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.351672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:06.356826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0beb86-54d7-4a5e-a42f-fa0e6e9558da None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:06.378424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40059025-80c3-4031-b13a-a21745fbb58a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:06.392236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee4e93-78c7-4b9a-8f6e-6464d86f4c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.392835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee4e93-78c7-4b9a-8f6e-6464d86f4c43 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.393110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee4e93-78c7-4b9a-8f6e-6464d86f4c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.393973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee4e93-78c7-4b9a-8f6e-6464d86f4c43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.406708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:06.414458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf91d38-e282-48aa-850b-6432fe4896f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.415069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91d38-e282-48aa-850b-6432fe4896f6 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.415349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91d38-e282-48aa-850b-6432fe4896f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.415694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf91d38-e282-48aa-850b-6432fe4896f6 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.416368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf91d38-e282-48aa-850b-6432fe4896f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.417789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf91d38-e282-48aa-850b-6432fe4896f6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.444133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 348/701] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:16:06 2022] POST /identity/v3/users => generated 254 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:06.444133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986153b7-05fb-4b28-90f4-1393c71bf226 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.444133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986153b7-05fb-4b28-90f4-1393c71bf226 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.444133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986153b7-05fb-4b28-90f4-1393c71bf226 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.444133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 349/702] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:06.444133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986153b7-05fb-4b28-90f4-1393c71bf226 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.487005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee4e93-78c7-4b9a-8f6e-6464d86f4c43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.488523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee4e93-78c7-4b9a-8f6e-6464d86f4c43 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.570475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986153b7-05fb-4b28-90f4-1393c71bf226 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:06.611432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:06.624155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.625661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.626143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.626786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.628026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:06.629161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.630263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.636228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 354/703] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 11 16:16:06.637585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218ea93-b9b9-42d5-a174-a1263d974001 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.637879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218ea93-b9b9-42d5-a174-a1263d974001 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.638151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218ea93-b9b9-42d5-a174-a1263d974001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.645149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 350/704] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:06.646353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.661644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8218ea93-b9b9-42d5-a174-a1263d974001 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:06.663256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8218ea93-b9b9-42d5-a174-a1263d974001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:06.714715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:06.755580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 355/705] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:06.756457 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218ea93-b9b9-42d5-a174-a1263d974001 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.792794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acc14e-12cf-4fca-b7bc-bdca41206436 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:06.828534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.829165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.829441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.829708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.840998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.842459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:06.858606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:06.859635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:06.860647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:06.860647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:06.860647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:06.860647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 351/706] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:06.861498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.886781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218ea93-b9b9-42d5-a174-a1263d974001 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:06.912492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:06.927995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:06.928647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:06.928925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:06.929196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:06.929929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=256cc27b87684392a22f36429b960644) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:06.930233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:06.931664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:06.944311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 356/707] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:06 2022] GET /identity/v3/role_assignments?user.id=256cc27b87684392a22f36429b960644&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:06.945411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:06.982584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2308e5-f73b-4184-bf7d-0e21b9960f22 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:07.022695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:07.023429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.023853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.024147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.024917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:07.026397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:07.027945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:07.043307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:07.045817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:07.047861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 352/708] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Aug 11 16:16:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:07.055553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:07.164600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2a415-4e67-4a53-b3c0-6bf242426508 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:07.207893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be422ac4-5169-44f5-a58b-3b64977e5e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:07.208560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be422ac4-5169-44f5-a58b-3b64977e5e17 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.208968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be422ac4-5169-44f5-a58b-3b64977e5e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.209394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be422ac4-5169-44f5-a58b-3b64977e5e17 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.210056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be422ac4-5169-44f5-a58b-3b64977e5e17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:07.210525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be422ac4-5169-44f5-a58b-3b64977e5e17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:07.212701 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be422ac4-5169-44f5-a58b-3b64977e5e17 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:07.213794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006bb10-e9bc-40bd-8541-ddd7267fc4cc None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:07.222880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:07.226333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 357/709] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:07.227019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2027d85-0c83-417d-ab93-9ee3d11f759e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.227304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2027d85-0c83-417d-ab93-9ee3d11f759e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.227577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2027d85-0c83-417d-ab93-9ee3d11f759e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.228982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 358/710] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:07.233386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2027d85-0c83-417d-ab93-9ee3d11f759e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:07.254428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22c6cb-163c-454f-89bd-64a1ab7328dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:07.255031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c6cb-163c-454f-89bd-64a1ab7328dd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.255320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c6cb-163c-454f-89bd-64a1ab7328dd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.255564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22c6cb-163c-454f-89bd-64a1ab7328dd None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.256228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22c6cb-163c-454f-89bd-64a1ab7328dd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:07.256580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22c6cb-163c-454f-89bd-64a1ab7328dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:07.257878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22c6cb-163c-454f-89bd-64a1ab7328dd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:07.263781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 353/711] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Aug 11 16:16:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:07.265044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ed224-601a-4c1b-be48-e7925fa6b9cb None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.265044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ed224-601a-4c1b-be48-e7925fa6b9cb None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.265315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ed224-601a-4c1b-be48-e7925fa6b9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.290263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-dc9ed224-601a-4c1b-be48-e7925fa6b9cb None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:07.291456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-dc9ed224-601a-4c1b-be48-e7925fa6b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:07.373181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:07.384265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 354/712] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:07.385098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ed224-601a-4c1b-be48-e7925fa6b9cb None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:07.389176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2027d85-0c83-417d-ab93-9ee3d11f759e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:07.478566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:07.479476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.479933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.480438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.491979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:07.494541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:07.515340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:07.517077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 359/713] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:07.522352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:07.548472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:07.554480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ed224-601a-4c1b-be48-e7925fa6b9cb None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:07.616484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:07.617942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.618771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.619486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.620857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:07.621662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:07.624408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:07.636117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 355/714] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Aug 11 16:16:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:07.637959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:07.699978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83b220-729f-4783-a156-0e9bddff7667 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:07.797982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:07.798894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.799394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.801018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] PATH_INFO: `/v3/services/network` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.801744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:07.804126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:07.826501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0dc974-3626-4b65-859b-1e5561929a11 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:07.844444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:07.848145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:07.850017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:07.860366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 360/715] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:07 2022] GET /identity/v3/services/network => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:07.863073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:07.900179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:07.901215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:07.903103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:07.904296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:07.905268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:07.905994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:07.910803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:07.924606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:07.936175 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 356/716] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:07.944141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:07.988211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:08.052207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74eb8ce-2998-4190-a374-12b594b42024 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:08.069609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346289e-afe2-4dc2-9334-9af9599e5c4b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:08.102790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:08.103606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:08.103981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:08.104370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:08.105190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] RBAC: Adding query filter params (name=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:08.105763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:08.107221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:08.112342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 361/717] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:08.117909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:08.136870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4e22c-703c-42f8-a82a-49370fc0ffb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:08.138140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4e22c-703c-42f8-a82a-49370fc0ffb9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:08.139743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4e22c-703c-42f8-a82a-49370fc0ffb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:08.140124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4e22c-703c-42f8-a82a-49370fc0ffb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:08.140976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4e22c-703c-42f8-a82a-49370fc0ffb9 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=2e3819e78802469c83c09a1cdd865f58) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:08.142565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4e22c-703c-42f8-a82a-49370fc0ffb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:08.145490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4e22c-703c-42f8-a82a-49370fc0ffb9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:08.156881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 357/718] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:07 2022] GET /identity/v3/role_assignments?user.id=2e3819e78802469c83c09a1cdd865f58&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=29fd031edd2b4f65aed8942853150f92 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:08.259425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:08.285325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84741e60-ab7f-4537-bad4-eb3d11450e39 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:08.331697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae72d54-01ce-45b2-88ff-772b4d5561a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:08.332667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae72d54-01ce-45b2-88ff-772b4d5561a9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:08.333199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae72d54-01ce-45b2-88ff-772b4d5561a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:08.333681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae72d54-01ce-45b2-88ff-772b4d5561a9 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:08.334502 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae72d54-01ce-45b2-88ff-772b4d5561a9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:08.335010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae72d54-01ce-45b2-88ff-772b4d5561a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:08.336694 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae72d54-01ce-45b2-88ff-772b4d5561a9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:08.343018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 362/719] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:08.798083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309852ff-8b90-44d9-90ee-1bdf28734bbf None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:08.798576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309852ff-8b90-44d9-90ee-1bdf28734bbf None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:08.798576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309852ff-8b90-44d9-90ee-1bdf28734bbf None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:08.800493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 358/720] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:08.805605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464297d-6271-4e95-80fd-dd85e4f5348d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:08.806013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464297d-6271-4e95-80fd-dd85e4f5348d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:08.806203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464297d-6271-4e95-80fd-dd85e4f5348d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:08.824538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6464297d-6271-4e95-80fd-dd85e4f5348d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:08.825707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6464297d-6271-4e95-80fd-dd85e4f5348d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:08.914269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 363/721] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:08.936634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309852ff-8b90-44d9-90ee-1bdf28734bbf None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:09.031815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:09.046134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309852ff-8b90-44d9-90ee-1bdf28734bbf None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:09.082935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b8220-9915-496a-baf0-62c2aa2b89ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:09.083567 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b8220-9915-496a-baf0-62c2aa2b89ad None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:09.083847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b8220-9915-496a-baf0-62c2aa2b89ad None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:09.084134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b8220-9915-496a-baf0-62c2aa2b89ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:09.095103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b8220-9915-496a-baf0-62c2aa2b89ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-149b8220-9915-496a-baf0-62c2aa2b89ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:09.099921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:09.100958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:09.101855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:09.105836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 359/722] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:16:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:09.107183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464297d-6271-4e95-80fd-dd85e4f5348d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:09.184866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:09.219965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464297d-6271-4e95-80fd-dd85e4f5348d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:09.264348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:09.264958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:09.265234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:09.265503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:09.266168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:09.266470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:09.267896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:09.283070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 364/723] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:16:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:09.288697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b8220-9915-496a-baf0-62c2aa2b89ad None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:09.398172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b8220-9915-496a-baf0-62c2aa2b89ad None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:09.433985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:09.434611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:09.434887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:09.435208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:09.448251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:09.449741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:09.457808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:09.464437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:09.465463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:09.466383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:09.466383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:09.466383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:09.469236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 360/724] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:09.471505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:09.603463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40c00e-42a4-4e28-a565-b7c024fe8dc1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:09.651733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42fd91-7efe-45f5-9938-c257ccba9425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:09.652288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42fd91-7efe-45f5-9938-c257ccba9425 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:09.652567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42fd91-7efe-45f5-9938-c257ccba9425 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:09.652836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42fd91-7efe-45f5-9938-c257ccba9425 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:09.653477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42fd91-7efe-45f5-9938-c257ccba9425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:09.653775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42fd91-7efe-45f5-9938-c257ccba9425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:09.655142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42fd91-7efe-45f5-9938-c257ccba9425 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:09.668475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:09.672957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 365/725] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:09.679507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:09.812592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1095b-d2bb-426e-9988-0f49cc3193e3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:09.836899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc805-3958-4530-a4e3-d0d8f19fa6d6 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:09.837392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc805-3958-4530-a4e3-d0d8f19fa6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:09.837862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc805-3958-4530-a4e3-d0d8f19fa6d6 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:09.841022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 366/726] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:09.845753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb790a7-4df5-4995-864c-13f3b44729ac None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:09.846051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb790a7-4df5-4995-864c-13f3b44729ac None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:09.846388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb790a7-4df5-4995-864c-13f3b44729ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:09.870702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6fb790a7-4df5-4995-864c-13f3b44729ac None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:09.871876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6fb790a7-4df5-4995-864c-13f3b44729ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:09.873269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:09.873896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:09.874165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:09.874430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:09.885004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:09.885927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:09.898157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:09.899151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:09.900149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:09.904547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 361/727] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:16:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:09.906067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:09.996939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 367/728] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 11 16:16:10.023340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb790a7-4df5-4995-864c-13f3b44729ac None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:10.024478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:10.114614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe14c1-7320-4236-9940-5f7d9219d7cf None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:10.176388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:10.176906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.177176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.177438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.178166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:10.178462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:10.181394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:10.190835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb790a7-4df5-4995-864c-13f3b44729ac None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:10.216379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 362/729] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Aug 11 16:16:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:10.217765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:10.269648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:10.285422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dca5c-33f2-46c8-a3d4-60b0a4315bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:10.286342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dca5c-33f2-46c8-a3d4-60b0a4315bfe None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.286773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dca5c-33f2-46c8-a3d4-60b0a4315bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.287194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dca5c-33f2-46c8-a3d4-60b0a4315bfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.320748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dca5c-33f2-46c8-a3d4-60b0a4315bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:10.321896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6fa46-21dd-4ee6-bfd2-2123d430a698 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-027dca5c-33f2-46c8-a3d4-60b0a4315bfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:10.326048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:10.327234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:10.328353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:10.330530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 368/730] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:10.344956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dca5c-33f2-46c8-a3d4-60b0a4315bfe None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:10.358706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:10.362802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc418b-a2e0-440f-9a95-15a21dfa36c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:10.363410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc418b-a2e0-440f-9a95-15a21dfa36c6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.363771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc418b-a2e0-440f-9a95-15a21dfa36c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.364037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc418b-a2e0-440f-9a95-15a21dfa36c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.436428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc418b-a2e0-440f-9a95-15a21dfa36c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:10.437786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc418b-a2e0-440f-9a95-15a21dfa36c6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:10.472331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dca5c-33f2-46c8-a3d4-60b0a4315bfe None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:10.511416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ddd89-6a73-4c52-b580-2d4890f4e137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:10.512290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ddd89-6a73-4c52-b580-2d4890f4e137 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.512715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ddd89-6a73-4c52-b580-2d4890f4e137 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.513156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ddd89-6a73-4c52-b580-2d4890f4e137 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.513985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ddd89-6a73-4c52-b580-2d4890f4e137 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:10.514434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ddd89-6a73-4c52-b580-2d4890f4e137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:10.515946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ddd89-6a73-4c52-b580-2d4890f4e137 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:10.525517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 369/731] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:10.527140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c000-9527-4741-9eba-5ff207a14fdc None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.527610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c000-9527-4741-9eba-5ff207a14fdc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.528062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c000-9527-4741-9eba-5ff207a14fdc None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.533550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 370/732] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:10.535595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940c000-9527-4741-9eba-5ff207a14fdc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:10.573102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 363/733] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 11 16:16:10.574956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa745cd-0d9a-41f1-94ab-be079620ac9a None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.575373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa745cd-0d9a-41f1-94ab-be079620ac9a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.575682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa745cd-0d9a-41f1-94ab-be079620ac9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.600354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:10.601413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-baa745cd-0d9a-41f1-94ab-be079620ac9a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:10.603750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-baa745cd-0d9a-41f1-94ab-be079620ac9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:10.621158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:10.665135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940c000-9527-4741-9eba-5ff207a14fdc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:10.692056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 364/734] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:10.693289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa745cd-0d9a-41f1-94ab-be079620ac9a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:10.709267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757727fd-3a34-41bf-94e4-dd9b5ac4a530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:10.709903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757727fd-3a34-41bf-94e4-dd9b5ac4a530 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.710183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757727fd-3a34-41bf-94e4-dd9b5ac4a530 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.710453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757727fd-3a34-41bf-94e4-dd9b5ac4a530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.723401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757727fd-3a34-41bf-94e4-dd9b5ac4a530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:10.724872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757727fd-3a34-41bf-94e4-dd9b5ac4a530 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-757727fd-3a34-41bf-94e4-dd9b5ac4a530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:10.740276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:10.741236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:10.742212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:10.742212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:10.742212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:10.744407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 371/735] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:10.745500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496fdbb-2444-4079-a345-999f1195a777 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.745853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496fdbb-2444-4079-a345-999f1195a777 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.746131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496fdbb-2444-4079-a345-999f1195a777 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.747528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 372/736] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:10.748061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496fdbb-2444-4079-a345-999f1195a777 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:10.815887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa745cd-0d9a-41f1-94ab-be079620ac9a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:10.851892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:10.852609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.852883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.853149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.855144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496fdbb-2444-4079-a345-999f1195a777 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:10.858273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:10.864163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:10.865630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:10.874769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:10.882050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:10.883219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:10.884888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:10.884888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:10.884888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:10.887088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 365/737] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:10.887725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:10.899118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d56eb-e7eb-4626-b565-8ab039c2a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:10.900510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d56eb-e7eb-4626-b565-8ab039c2a19b None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.901024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d56eb-e7eb-4626-b565-8ab039c2a19b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.901896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d56eb-e7eb-4626-b565-8ab039c2a19b None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.903031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d56eb-e7eb-4626-b565-8ab039c2a19b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:10.905270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d56eb-e7eb-4626-b565-8ab039c2a19b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:10.925179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 373/738] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:16:10 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:10.926471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be03469-c392-4261-a6bd-8a3e3fb11575 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:10.926764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be03469-c392-4261-a6bd-8a3e3fb11575 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:10.927034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be03469-c392-4261-a6bd-8a3e3fb11575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:10.946043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2be03469-c392-4261-a6bd-8a3e3fb11575 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:10.947198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2be03469-c392-4261-a6bd-8a3e3fb11575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:11.009837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb0f69-cbd9-44c6-9bc8-58dc59135ba9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.052126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 374/739] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:11.052126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be03469-c392-4261-a6bd-8a3e3fb11575 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.067054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c67fd-1527-4d63-8878-ac05b6df6c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.067969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c67fd-1527-4d63-8878-ac05b6df6c51 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.068285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c67fd-1527-4d63-8878-ac05b6df6c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.068564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c67fd-1527-4d63-8878-ac05b6df6c51 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.069559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c67fd-1527-4d63-8878-ac05b6df6c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:11.070292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c67fd-1527-4d63-8878-ac05b6df6c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.071809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c67fd-1527-4d63-8878-ac05b6df6c51 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.087539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 366/740] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:11.088916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a041-e8d8-4cd9-816a-0885f1e796c2 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.089234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a041-e8d8-4cd9-816a-0885f1e796c2 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.089500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a041-e8d8-4cd9-816a-0885f1e796c2 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.090990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 367/741] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:11.091788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71a041-e8d8-4cd9-816a-0885f1e796c2 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.153207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:11.163768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:11.165486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be03469-c392-4261-a6bd-8a3e3fb11575 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.202599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.203145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.203406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.203675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.204352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:11.204656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.206652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.207611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71a041-e8d8-4cd9-816a-0885f1e796c2 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.220481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 375/742] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:11.225513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.243457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410603bd-6474-4b26-9d8a-2862fdff034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.244090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410603bd-6474-4b26-9d8a-2862fdff034c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.244432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410603bd-6474-4b26-9d8a-2862fdff034c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.244717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410603bd-6474-4b26-9d8a-2862fdff034c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.255661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410603bd-6474-4b26-9d8a-2862fdff034c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-410603bd-6474-4b26-9d8a-2862fdff034c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:11.260790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:11.261741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:11.262621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 368/743] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:11.263320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafc4c-c461-4b75-b3c6-5de1ccca52e7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.263389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafc4c-c461-4b75-b3c6-5de1ccca52e7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.264121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafc4c-c461-4b75-b3c6-5de1ccca52e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.281845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-a5dafc4c-c461-4b75-b3c6-5de1ccca52e7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:11.283048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a5dafc4c-c461-4b75-b3c6-5de1ccca52e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:11.340386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd2f47-11b2-4936-a8ce-cd375fbb3b08 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.356924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 369/744] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 11 16:16:11.358358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dafc4c-c461-4b75-b3c6-5de1ccca52e7 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.381497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.382159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.382432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.382704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.394819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.396295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:11.407894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:11.409195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 376/745] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:11.410210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.422463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:11.425576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:11.480325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dafc4c-c461-4b75-b3c6-5de1ccca52e7 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.516173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.516798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.517126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.517346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.525040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.526780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.534161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d70c46-5db6-4d4c-94a3-fcfff5bb34bd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:11.537675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:11.538660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:11.539571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:11.539571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:11.539571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:11.539571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:11.539571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:11.539571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:11.539571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:11.539571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:11.543445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 370/746] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:11.544274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.569477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.570081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.570362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.570631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.571292 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:11.571597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.572984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.586178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 377/747] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:11.586178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.657060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:11.658958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b10b7c-ac77-4099-ab3c-a213ddb5f9cd None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.670596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:11.692326 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75744074-81ff-4e4e-8dd2-7cc3c24a9bc0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.697797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9d466-d312-4280-8c83-512ceb99ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.698400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d466-d312-4280-8c83-512ceb99ccf0 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.698676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d466-d312-4280-8c83-512ceb99ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.698943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d466-d312-4280-8c83-512ceb99ccf0 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.699582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9d466-d312-4280-8c83-512ceb99ccf0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.700961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9d466-d312-4280-8c83-512ceb99ccf0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.719144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 371/748] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:16:11 2022] POST /identity/v3/services => generated 240 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:11.720840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9d466-d312-4280-8c83-512ceb99ccf0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.735325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.735957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.736381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.736707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.738274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:11.738732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.740332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.764142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 378/749] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:11.764142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.856455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9d466-d312-4280-8c83-512ceb99ccf0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.900898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.902049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.902334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.902606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.903498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:11.904190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.905914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.906459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5d60e-b945-4c6f-ae7f-3c3806e5b0b2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:11.921301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 372/750] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:11.922444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:11.942478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:11.954265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:11.955479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:11.956478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:11.956760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:11.967029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:11.968505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:11.978039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:11.984550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:11.985557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:11.986475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:11.986475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:11.986475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:11.986475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 379/751] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:11.987408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:12.077868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eabcb3-cb63-4e70-ae25-245795ac2df3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:12.117328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926efeae-053d-4abd-9fec-9e4ddc014d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:12.117711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926efeae-053d-4abd-9fec-9e4ddc014d80 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:12.118102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926efeae-053d-4abd-9fec-9e4ddc014d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:12.118377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926efeae-053d-4abd-9fec-9e4ddc014d80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:12.129075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835d057-318c-426f-85be-950cf95184b7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:12.182449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:12.183562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:12.184426 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:12.185112 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:12.186235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=d7291ccb171e44f89dabc99980312c3b) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:12.187108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:12.189494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:12.197720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926efeae-053d-4abd-9fec-9e4ddc014d80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:12.198140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 380/752] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/role_assignments?user.id=d7291ccb171e44f89dabc99980312c3b&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:12.199037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926efeae-053d-4abd-9fec-9e4ddc014d80 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:12.199328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:12.234575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:12.262700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:12.338748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 373/753] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:16:12.343884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926efeae-053d-4abd-9fec-9e4ddc014d80 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:12.361252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa10bd-abf1-47a4-a1e3-aa59b2478b15 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:12.430686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:12.431318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:12.431592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:12.432811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:12.432811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:12.432993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:12.435647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:12.453769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 381/754] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:12.461213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:12.504951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926efeae-053d-4abd-9fec-9e4ddc014d80 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:12.550140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:12.572816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:12.573587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:12.573942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:12.574290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:12.581580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e2c6d-8bc7-488a-aac0-357a89fcb882 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:12.583420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:12.597238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:12.598899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:12.622935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:12.623971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:12.624455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:12.624930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:12.632914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:12.633069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:12.633978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:12.634943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:12.634943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:12.634943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:12.634943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 374/755] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:12.636999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:12.639728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:12.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:12.653364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:12.654577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:12.657058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 382/756] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:16:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:12.662749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:12.792182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4fa08e-3a50-4fc0-8d80-b60a898d9f17 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:12.831256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:12.832761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:12.833184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:12.833593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:12.834466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:12.834932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:12.836528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:12.864517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 383/757] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:16:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:12.866596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:12.889413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:12.950278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfd112-6e2b-4c74-a3d1-da92d7652c32 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:12.976499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366bbb7-888e-49dc-b7a7-48983891c30c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.015215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:13.015964 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.016308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.016665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.022763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:13.023501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.024329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.024723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.025896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:13.026310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:13.027571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:13.028234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:13.029056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:13.047984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 375/758] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:13.048888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:13.050761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:13.051679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:13.052705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:13.052705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:13.052705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:13.052705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 384/759] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:13.066199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:13.133883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:13.172462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:13.228692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89ef40-3db6-41ee-9fd9-35979c8839d0 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.239477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c13f5e-7fbf-45a6-9586-658943f67ed6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.273737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5945c-c1e2-4641-a7b0-de798be7d9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:13.274565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5945c-c1e2-4641-a7b0-de798be7d9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.274684 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:13.275105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5945c-c1e2-4641-a7b0-de798be7d9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.275289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.275568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.275836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.275962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5945c-c1e2-4641-a7b0-de798be7d9d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.276760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5945c-c1e2-4641-a7b0-de798be7d9d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:13.277207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5945c-c1e2-4641-a7b0-de798be7d9d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:13.279402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5945c-c1e2-4641-a7b0-de798be7d9d2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:13.282623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:13.284130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:13.293346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 385/760] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:13.294700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3dd86-71d0-4da1-ab4e-f3ffd5538185 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.295157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3dd86-71d0-4da1-ab4e-f3ffd5538185 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:13.295274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:13.296382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:13.297603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:13.297603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:13.297603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:13.297603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:13.297603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:13.297603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:13.297603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:13.297603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:13.298518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3dd86-71d0-4da1-ab4e-f3ffd5538185 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.301481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 386/761] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:13.301774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 376/762] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:13.302566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3dd86-71d0-4da1-ab4e-f3ffd5538185 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:13.308147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:13.418980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93514fec-9e37-4ca6-ac70-395e47a0151b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.420408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:13.437973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3dd86-71d0-4da1-ab4e-f3ffd5538185 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.466709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4806a-191d-47af-babd-92b61b5f3fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:13.467598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4806a-191d-47af-babd-92b61b5f3fad None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.468030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4806a-191d-47af-babd-92b61b5f3fad None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.468476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4806a-191d-47af-babd-92b61b5f3fad None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.469380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4806a-191d-47af-babd-92b61b5f3fad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:13.469891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4806a-191d-47af-babd-92b61b5f3fad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:13.471776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4806a-191d-47af-babd-92b61b5f3fad None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:13.477011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:13.484148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:13.485078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.485365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.485634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.488559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 377/763] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:13.490043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eef97f-fc5e-4564-93e6-19031dd4c076 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.490494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eef97f-fc5e-4564-93e6-19031dd4c076 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.490911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eef97f-fc5e-4564-93e6-19031dd4c076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.493929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:13.495379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:13.510102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:13.511247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 387/764] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:13.512442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:13.514009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-51eef97f-fc5e-4564-93e6-19031dd4c076 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:13.515377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-51eef97f-fc5e-4564-93e6-19031dd4c076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:13.607067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 378/765] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:13.608000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eef97f-fc5e-4564-93e6-19031dd4c076 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:13.638630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c95fd-fca7-4034-911b-1a637487f3cc None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.678528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:13.679428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.679959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.680423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.681318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=256cc27b87684392a22f36429b960644) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:13.681812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:13.683439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:13.690056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 388/766] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:13 2022] GET /identity/v3/role_assignments?user.id=256cc27b87684392a22f36429b960644&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=29fd031edd2b4f65aed8942853150f92 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:13.691111 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:13.730175 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:13.743675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eef97f-fc5e-4564-93e6-19031dd4c076 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.786997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:13.788452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:13.789015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:13.789438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:13.790319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:13.790897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:13.792929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:13.810274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:13.811686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 379/767] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:13.818480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:13.949715 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b146016-d324-4e81-9a32-9a435457b16f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.992854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dde6e-5fdf-4882-a438-ea6f96be3d71 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:13.999306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eb2e2-a9ad-4482-9f6e-efcd1ba7c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:14.000548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb2e2-a9ad-4482-9f6e-efcd1ba7c603 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.001026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb2e2-a9ad-4482-9f6e-efcd1ba7c603 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.001511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb2e2-a9ad-4482-9f6e-efcd1ba7c603 None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/2e3819e78802469c83c09a1cdd865f58/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.010342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eb2e2-a9ad-4482-9f6e-efcd1ba7c603 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=2e3819e78802469c83c09a1cdd865f58, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:14.012113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eb2e2-a9ad-4482-9f6e-efcd1ba7c603 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:14.032933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 380/768] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:16:13 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/2e3819e78802469c83c09a1cdd865f58/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:16:14.034899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cecb-6bc0-4ad3-bfb6-8ef573853717 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.035102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cecb-6bc0-4ad3-bfb6-8ef573853717 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.035379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718cecb-6bc0-4ad3-bfb6-8ef573853717 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.044539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 381/769] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:14.059409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52e384-b467-41ea-b00a-c2de9b90b6f2 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.060460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52e384-b467-41ea-b00a-c2de9b90b6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.061507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52e384-b467-41ea-b00a-c2de9b90b6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.062756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:14.064799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.066371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.066869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.067731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:14.071189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:14.091664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:14.093053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:14.094222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:14.099065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 389/770] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Aug 11 16:16:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:14.104495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:14.111492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ec52e384-b467-41ea-b00a-c2de9b90b6f2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:14.113698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ec52e384-b467-41ea-b00a-c2de9b90b6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:14.133496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:14.262177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 382/771] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:14.265648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcccad-864e-4652-84e6-cf9bc88062df None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:14.288286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52e384-b467-41ea-b00a-c2de9b90b6f2 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:14.290461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:14.307075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:14.307809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.308014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.308294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.308923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:14.309220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:14.310590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:14.317325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 390/772] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Aug 11 16:16:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:14.328597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:14.429866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915956ca-fed5-407c-a74a-9b15636e112e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:14.465857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:14.466467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.466744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.467013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.467644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:14.467938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:14.469373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:14.475570 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 391/773] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Aug 11 16:16:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:14.483917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:14.506056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52e384-b467-41ea-b00a-c2de9b90b6f2 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:14.544569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:14.591139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507c38d-3530-4984-aa9f-1f1bbc0d2785 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:14.628314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecde8c-64b7-47d5-9d5e-429dd83e6029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:14.628989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecde8c-64b7-47d5-9d5e-429dd83e6029 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.629265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecde8c-64b7-47d5-9d5e-429dd83e6029 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.629573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecde8c-64b7-47d5-9d5e-429dd83e6029 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.630280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecde8c-64b7-47d5-9d5e-429dd83e6029 None admin] RBAC: Adding query filter params (interface=public, service_id=51ec425b397144818dddb3cae1656bbf, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:14.630581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecde8c-64b7-47d5-9d5e-429dd83e6029 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:14.631936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecde8c-64b7-47d5-9d5e-429dd83e6029 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:14.639197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbaf65-044c-4126-8cb0-9ea8ab36f555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:14.639838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbaf65-044c-4126-8cb0-9ea8ab36f555 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.640128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbaf65-044c-4126-8cb0-9ea8ab36f555 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.640407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbaf65-044c-4126-8cb0-9ea8ab36f555 None admin] PATH_INFO: `/v3/services/network` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.640658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 392/774] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Aug 11 16:16:14 2022] GET /identity/v3/endpoints?service_id=51ec425b397144818dddb3cae1656bbf&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:14.641068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbaf65-044c-4126-8cb0-9ea8ab36f555 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:14.641858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d38871-17a6-42f0-b204-ba3856bdf06f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.642136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d38871-17a6-42f0-b204-ba3856bdf06f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.642406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d38871-17a6-42f0-b204-ba3856bdf06f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.642568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbaf65-044c-4126-8cb0-9ea8ab36f555 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:14.644416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 393/775] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:14.646255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d38871-17a6-42f0-b204-ba3856bdf06f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-d0dbaf65-044c-4126-8cb0-9ea8ab36f555 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:14.656769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:14.657805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:14.659280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 383/776] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:14 2022] GET /identity/v3/services/network => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:14.660193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26490f01-8e20-4df6-a260-ca002fcf95d8 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.660890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26490f01-8e20-4df6-a260-ca002fcf95d8 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.661052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26490f01-8e20-4df6-a260-ca002fcf95d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.674674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:14.681025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-26490f01-8e20-4df6-a260-ca002fcf95d8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:14.682277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-26490f01-8e20-4df6-a260-ca002fcf95d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:14.765275 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d38871-17a6-42f0-b204-ba3856bdf06f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:14.766553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 384/777] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:14.767586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26490f01-8e20-4df6-a260-ca002fcf95d8 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:14.790037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:14.826997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08e667-b92f-4884-9586-9bbeaeb556c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:14.827604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e667-b92f-4884-9586-9bbeaeb556c5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.827879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e667-b92f-4884-9586-9bbeaeb556c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.828174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e667-b92f-4884-9586-9bbeaeb556c5 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.829023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08e667-b92f-4884-9586-9bbeaeb556c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:14.830342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08e667-b92f-4884-9586-9bbeaeb556c5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:14.837043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 394/778] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:16:14 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:14.838252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08e667-b92f-4884-9586-9bbeaeb556c5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:14.888502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26490f01-8e20-4df6-a260-ca002fcf95d8 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:14.928732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:14.929742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:14.930206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:14.930661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:14.931589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:14.932216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:14.933936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:14.938021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:14.946791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 385/779] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:14.948173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:14.981595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08e667-b92f-4884-9586-9bbeaeb556c5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:15.019404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a312a-107b-4d2a-873d-c3c5217c5954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:15.020269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a312a-107b-4d2a-873d-c3c5217c5954 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:15.020455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a312a-107b-4d2a-873d-c3c5217c5954 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:15.020659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a312a-107b-4d2a-873d-c3c5217c5954 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:15.032764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a312a-107b-4d2a-873d-c3c5217c5954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-683a312a-107b-4d2a-873d-c3c5217c5954 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:15.038072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:15.039159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:15.040032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:15.044284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 395/780] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:16:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:15.046835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a312a-107b-4d2a-873d-c3c5217c5954 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:15.083322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a4191-876d-41dc-be29-6a9c0ef171a7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:15.084213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:15.126289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:15.127180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:15.128215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:15.128638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:15.129446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:15.129996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:15.131646 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:15.138086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 386/781] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:15.145889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:15.154902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a312a-107b-4d2a-873d-c3c5217c5954 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:15.189329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:15.190561 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:15.190973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:15.191238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:15.191899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:15.192224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:15.193590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:15.204212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 396/782] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:16:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:15.208239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:15.238931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:15.260973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed42425-1320-469c-98f3-043db679b4d2 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:15.298861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:15.299957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:15.300453 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:15.300909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:15.301800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] RBAC: Adding query filter params (interface=public, service_id=4bcc32c8cd9c43a288fa807ae497bf0f, region_id=RegionOne) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:15.302391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:15.304029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:15.311144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:15.315533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4ecf5-09e6-446d-9111-f2d7e0e9577e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:15.320211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 387/783] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Aug 11 16:16:15 2022] GET /identity/v3/endpoints?service_id=4bcc32c8cd9c43a288fa807ae497bf0f&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:15.325352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:15.354379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:15.354936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:15.355211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:15.355480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:15.366968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:15.368425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:15.383483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:15.384539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:15.385468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:15.385468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:15.385468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:15.389538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 397/784] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:15.391165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:15.444439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33901919-bb04-4959-9dfc-ae23f7f1d21e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:15.488809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:15.500029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfe4c7-5a4f-46db-89ed-9d4e9cfdbea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:15.501001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfe4c7-5a4f-46db-89ed-9d4e9cfdbea8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:15.501698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfe4c7-5a4f-46db-89ed-9d4e9cfdbea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:15.503621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfe4c7-5a4f-46db-89ed-9d4e9cfdbea8 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:15.505133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfe4c7-5a4f-46db-89ed-9d4e9cfdbea8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:15.508634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfe4c7-5a4f-46db-89ed-9d4e9cfdbea8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:15.519992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 388/785] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:16:15 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:15.548742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97ff9b-f326-4fab-b7e3-4e9b8aebbaa9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:15.593100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:15.593723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:15.593997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:15.594265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:15.594926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:15.595224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:15.597227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:15.617518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 398/786] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:15.623558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfe4c7-5a4f-46db-89ed-9d4e9cfdbea8 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:15.760402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfe4c7-5a4f-46db-89ed-9d4e9cfdbea8 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:15.797458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:15.798088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:15.798355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:15.798690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:15.809678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:15.811122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:15.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:15.824356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:15.825343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:15.828201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 389/787] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:16:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:15.830708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:15.852066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:15.982211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe198f9-ee0e-475f-bb6e-552574277170 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:16.018658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78044ca0-e445-48d5-94dd-dbce4e82608c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:16.019278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044ca0-e445-48d5-94dd-dbce4e82608c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.019556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044ca0-e445-48d5-94dd-dbce4e82608c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.020079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044ca0-e445-48d5-94dd-dbce4e82608c None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.020631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044ca0-e445-48d5-94dd-dbce4e82608c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:16.020932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044ca0-e445-48d5-94dd-dbce4e82608c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:16.022365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044ca0-e445-48d5-94dd-dbce4e82608c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:16.040822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 399/788] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Aug 11 16:16:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:16.046511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:16.130854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:16.196272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10bd3d-be1e-4c10-95ac-9f23831b680c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:16.227064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f145ca-50ef-4549-a901-50b8cd753f88 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.227742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f145ca-50ef-4549-a901-50b8cd753f88 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.228217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f145ca-50ef-4549-a901-50b8cd753f88 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.229726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 400/789] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:16.235074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cbb2a-4dde-4160-b756-d4258fbf0dd5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.235595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cbb2a-4dde-4160-b756-d4258fbf0dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.236067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cbb2a-4dde-4160-b756-d4258fbf0dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.252039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:16.253381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.253659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.253934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.257284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bd3cbb2a-4dde-4160-b756-d4258fbf0dd5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:16.258685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bd3cbb2a-4dde-4160-b756-d4258fbf0dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:16.278319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:16.281481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:16.309405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:16.310429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:16.311338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:16.311338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:16.311338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:16.316498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 390/790] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:16.324680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:16.380456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:16.389192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 401/791] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:16.435983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cbb2a-4dde-4160-b756-d4258fbf0dd5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:16.530411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01b4a6-bbfc-4fce-a847-2ed28fe7a0f9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:16.537174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:16.573733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cbb2a-4dde-4160-b756-d4258fbf0dd5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:16.615995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaca45a-e0e5-49fd-b725-752b3a79766b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:16.617053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca45a-e0e5-49fd-b725-752b3a79766b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.617528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca45a-e0e5-49fd-b725-752b3a79766b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.617992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaca45a-e0e5-49fd-b725-752b3a79766b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.626967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:16.628506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.629950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.630580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.632349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:16.633744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:16.636915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:16.640276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaca45a-e0e5-49fd-b725-752b3a79766b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6eaca45a-e0e5-49fd-b725-752b3a79766b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:16.651473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:16.652475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:16.653882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:16.657762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 402/792] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:16.659546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12537c1a-9113-4945-a1dc-5d72b669529c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.660115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12537c1a-9113-4945-a1dc-5d72b669529c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.660486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12537c1a-9113-4945-a1dc-5d72b669529c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.665347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 391/794] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:16.665654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 403/794] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:16.666668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:16.667034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd230e3e-358a-4b75-8528-b078d4d512b2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.667525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd230e3e-358a-4b75-8528-b078d4d512b2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.668037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd230e3e-358a-4b75-8528-b078d4d512b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.687898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-dd230e3e-358a-4b75-8528-b078d4d512b2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:16.690878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dd230e3e-358a-4b75-8528-b078d4d512b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:16.760475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:16.780185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 404/795] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:16.781287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd230e3e-358a-4b75-8528-b078d4d512b2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:16.805004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d6745-5400-48e9-a6c7-5888ab03c559 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:16.848995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:16.850312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.850769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.851182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.852404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:16.853117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:16.854920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:16.867844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 392/796] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:16.869306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:16.903485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:16.933849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd230e3e-358a-4b75-8528-b078d4d512b2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:16.972442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:16.973121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:16.973399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:16.973666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:16.980855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:16.982127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:16.993529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:16.994480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:16.995395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:16.995395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:16.995395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:16.995395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:16.995395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:16.995395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:16.995395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:16.995395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:16.999334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 405/797] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:17.000194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.008182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f61f12-dba0-4b1a-a76e-d8f81b162da2 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.052872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5c310-ab4d-4450-93f3-d45d002e1e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.053833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5c310-ab4d-4450-93f3-d45d002e1e4d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.054332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5c310-ab4d-4450-93f3-d45d002e1e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.055007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5c310-ab4d-4450-93f3-d45d002e1e4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.065655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:17.072492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5c310-ab4d-4450-93f3-d45d002e1e4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-81c5c310-ab4d-4450-93f3-d45d002e1e4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:17.081256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:17.082204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:17.083046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:17.090844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 393/798] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:17.090844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5c310-ab4d-4450-93f3-d45d002e1e4d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.129375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11dbac-4149-4a3c-8229-cbb7c52d5a58 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.175258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.176355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.176833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.177297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.190712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:17.194491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:17.196802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:17.208316 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5c310-ab4d-4450-93f3-d45d002e1e4d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:17.217387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:17.218330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:17.219221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:17.219221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:17.219221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:17.222823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 406/799] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:17.224146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.247629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.248312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.248599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.248949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.249631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:17.249936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:17.251427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:17.266583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 394/800] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:17.267672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.363354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:17.368940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447518-bbf5-4eb2-925d-628f3c7f06dd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.376350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caffe28-4467-4825-97f4-42a32f96916f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.413661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.414301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.414575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.414847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.415498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:17.415799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:17.417194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:17.418812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.420206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.420706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.421203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.422061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:17.422575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:17.425205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:17.435076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 395/801] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:17.441245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 407/802] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:17.441544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.441670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.452582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:17.548695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39abf72-2a62-4032-a87c-e7910312e43e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.552388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57942e3d-c0de-4552-8c76-856b6b4c32c7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.584708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84697f4a-cce8-4a59-8ca9-8846f495ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.585365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697f4a-cce8-4a59-8ca9-8846f495ef59 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.585658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697f4a-cce8-4a59-8ca9-8846f495ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.585910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84697f4a-cce8-4a59-8ca9-8846f495ef59 None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/d7291ccb171e44f89dabc99980312c3b/roles/7bd0567f3a3c43628a4ec259a52f1e24` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.588860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5f8a8-5155-479f-8c67-18db44290183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.589695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f8a8-5155-479f-8c67-18db44290183 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.590048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f8a8-5155-479f-8c67-18db44290183 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.590322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5f8a8-5155-479f-8c67-18db44290183 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.598210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5f8a8-5155-479f-8c67-18db44290183 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:17.599641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5f8a8-5155-479f-8c67-18db44290183 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:17.601373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84697f4a-cce8-4a59-8ca9-8846f495ef59 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=d7291ccb171e44f89dabc99980312c3b, role_id=7bd0567f3a3c43628a4ec259a52f1e24)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:17.602807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84697f4a-cce8-4a59-8ca9-8846f495ef59 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6ae5f8a8-5155-479f-8c67-18db44290183 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:17.612702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:17.613702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:16:17.614718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:17.616136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 396/803] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:16:17 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/d7291ccb171e44f89dabc99980312c3b/roles/7bd0567f3a3c43628a4ec259a52f1e24 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:16:17.620145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84697f4a-cce8-4a59-8ca9-8846f495ef59 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.621866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 408/804] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:17.623604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1460-45f7-45d3-a7de-f5f5f6cd98db None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.623885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1460-45f7-45d3-a7de-f5f5f6cd98db None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.624184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1460-45f7-45d3-a7de-f5f5f6cd98db None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.630575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 409/805] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:17.631534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d1460-45f7-45d3-a7de-f5f5f6cd98db None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.659242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:17.699738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:17.766805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d1460-45f7-45d3-a7de-f5f5f6cd98db None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.767302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84697f4a-cce8-4a59-8ca9-8846f495ef59 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.811711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c45cc-0dde-42ae-924f-90c6442266a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.812354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c45cc-0dde-42ae-924f-90c6442266a2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.812627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c45cc-0dde-42ae-924f-90c6442266a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.812893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c45cc-0dde-42ae-924f-90c6442266a2 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.813884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c45cc-0dde-42ae-924f-90c6442266a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:17.814181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c45cc-0dde-42ae-924f-90c6442266a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:17.816905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:17.817668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.817935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c45cc-0dde-42ae-924f-90c6442266a2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:17.818188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.818511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.830992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:17.832754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:17.839737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 410/806] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Aug 11 16:16:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:17.841002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3371-d413-4af3-ac92-64e8279ae45d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:17.841284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3371-d413-4af3-ac92-64e8279ae45d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:17.841556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3371-d413-4af3-ac92-64e8279ae45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:17.849726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:17.850902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:17.852030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:17.852030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:17.852030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:17.852030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 397/807] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:17.856590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.863531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-720e3371-d413-4af3-ac92-64e8279ae45d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:17.865084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-720e3371-d413-4af3-ac92-64e8279ae45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:17.935658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 411/808] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:17.944310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3371-d413-4af3-ac92-64e8279ae45d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:17.952432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:17.973956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb44c6-7542-40d4-b9bd-d72ded4b746d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:17.984735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:18.013755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1b463-6c75-4d22-9565-952ba04014ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.014333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b463-6c75-4d22-9565-952ba04014ae None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.014618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b463-6c75-4d22-9565-952ba04014ae None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.014950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b463-6c75-4d22-9565-952ba04014ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.028492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1b463-6c75-4d22-9565-952ba04014ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.030064 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1b463-6c75-4d22-9565-952ba04014ae None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-4be1b463-6c75-4d22-9565-952ba04014ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:18.050060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:18.051331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:18.052556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:18.052556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:18.052556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:18.060152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 398/809] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:18.060313 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3371-d413-4af3-ac92-64e8279ae45d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.060416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca08b1-9026-4737-adfa-d3f85a5cff92 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.060416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca08b1-9026-4737-adfa-d3f85a5cff92 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.060416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca08b1-9026-4737-adfa-d3f85a5cff92 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.060416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 399/810] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:18.060416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca08b1-9026-4737-adfa-d3f85a5cff92 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.091610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.092829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.093268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.093681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.094485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:18.094932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.097054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.108937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 412/811] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:18.109931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.182451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca08b1-9026-4737-adfa-d3f85a5cff92 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.184308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:18.218247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66342035-4b14-40cf-a4d1-bc2b6a899891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.218966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342035-4b14-40cf-a4d1-bc2b6a899891 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.219137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342035-4b14-40cf-a4d1-bc2b6a899891 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.219383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d1cc6-a834-4621-b0c4-b5a8ee1dae59 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.219490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342035-4b14-40cf-a4d1-bc2b6a899891 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.219975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66342035-4b14-40cf-a4d1-bc2b6a899891 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.222208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66342035-4b14-40cf-a4d1-bc2b6a899891 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-66342035-4b14-40cf-a4d1-bc2b6a899891 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:18.231818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:18.232867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:18.233719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 400/812] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:18.234531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b406df-53ca-476c-a009-637fac9db585 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.234531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b406df-53ca-476c-a009-637fac9db585 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.235202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b406df-53ca-476c-a009-637fac9db585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.252955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d8b406df-53ca-476c-a009-637fac9db585 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:18.254067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d8b406df-53ca-476c-a009-637fac9db585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:18.261103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.262257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.262644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.263023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.264185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:18.264596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.266389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.272079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:18.279867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 413/813] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:18.280719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.328409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 401/814] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:18.329585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b406df-53ca-476c-a009-637fac9db585 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.401665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f60acf-45c4-4c0a-be09-4f219ca0a966 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.436178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:18.437897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b406df-53ca-476c-a009-637fac9db585 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.444150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.444787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.445065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.445337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.453420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.455988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:18.469154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:18.470138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:18.471703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:18.478179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 414/815] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:18.478179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.479371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.479985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.480288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.480580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.481213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:18.481512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.482864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.487831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 402/816] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:18.492972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.513876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:18.579385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4e544-d5a0-437c-bd4d-47e4f52668cb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.599057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590c32-d65d-4777-93e4-05a0fde47433 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.615126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.615864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.616277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.616553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.624609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.626174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.632465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.633081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.633356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.633621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] PATH_INFO: `/v3/services/network` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.634193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.635520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:18.638076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:18.638994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 415/817] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:18.639904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:18.649839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:18.650975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:18.651823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 403/818] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:18.652682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.702099 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:18.748116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:18.762341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbec83-6344-4e3c-8dc7-235ce48040ae None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.767044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a160f4c-8c52-4bb6-89f2-97c0e29e68ee None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.800348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.800956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.801390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.801663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.802971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:18.803283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.804747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.806912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.807512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.807793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.808072 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.808750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:18.809051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.810407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.815626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 404/819] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:18.816462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.836237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 416/820] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:18.837463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.926024 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4081-aaf0-4499-9617-2b440a51f9d0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.944996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb045ba-89da-46a6-85c5-7437d3a1a832 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:18.955040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:18.963776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.964430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.964712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.964982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.965716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:18.966015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.967442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.981183 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 405/821] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:18.982378 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.982582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:18.983156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:18.983412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:18.983696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:18.984344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] RBAC: Adding query filter params (name=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:18.984646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:18.985992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:18.990930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 417/822] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:18.991707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:18.996949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:19.099210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51f18a-ee75-460e-8a71-47f4f826f556 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:19.101849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b548e4-2152-416e-ad40-2802a1edadff None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:19.135039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b956e-a7fe-4ede-aa38-6bef0ad27ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:19.135731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b956e-a7fe-4ede-aa38-6bef0ad27ba4 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.136075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b956e-a7fe-4ede-aa38-6bef0ad27ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.136453 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b956e-a7fe-4ede-aa38-6bef0ad27ba4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.137184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b956e-a7fe-4ede-aa38-6bef0ad27ba4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:19.138615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b956e-a7fe-4ede-aa38-6bef0ad27ba4 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:19.140748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f92b-d0af-46e2-9323-c89aa5b6d65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:19.141455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f92b-d0af-46e2-9323-c89aa5b6d65d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.141740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f92b-d0af-46e2-9323-c89aa5b6d65d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.142016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f92b-d0af-46e2-9323-c89aa5b6d65d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.164678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 406/823] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:16:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:19.165665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b956e-a7fe-4ede-aa38-6bef0ad27ba4 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:19.203052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:19.239129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070f92b-d0af-46e2-9323-c89aa5b6d65d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:19.240831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070f92b-d0af-46e2-9323-c89aa5b6d65d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:19.272980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:19.349994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b956e-a7fe-4ede-aa38-6bef0ad27ba4 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:19.358462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 418/824] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:16:19.359987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f92b-d0af-46e2-9323-c89aa5b6d65d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:19.393677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6b2bf-28f6-4ec0-8bb9-093eb4e083e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:19.394346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6b2bf-28f6-4ec0-8bb9-093eb4e083e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.394625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6b2bf-28f6-4ec0-8bb9-093eb4e083e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.394901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6b2bf-28f6-4ec0-8bb9-093eb4e083e0 None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/256cc27b87684392a22f36429b960644/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.396817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6b2bf-28f6-4ec0-8bb9-093eb4e083e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=256cc27b87684392a22f36429b960644, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:19.398255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6b2bf-28f6-4ec0-8bb9-093eb4e083e0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:19.415313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 407/825] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:16:19 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/256cc27b87684392a22f36429b960644/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:16:19.417454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6b2bf-28f6-4ec0-8bb9-093eb4e083e0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:19.446692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:19.489575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f92b-d0af-46e2-9323-c89aa5b6d65d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:19.530678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:19.531013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.531354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.531638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.532318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] RBAC: Adding query filter params (type=network) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:19.532622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:19.534058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:19.543373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 419/826] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:19.545970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:19.565877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6b2bf-28f6-4ec0-8bb9-093eb4e083e0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:19.589979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:19.616271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:19.617173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.617527 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.617865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.632249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:19.634115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:19.651096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:19.652180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:19.653117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:19.653117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:19.653117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:19.656447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 408/827] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:19.659473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:19.683545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ca234-c053-4d0b-9428-9e0660415a41 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:19.725529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af22005-e61d-4c5e-8220-b02a7fc75c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:19.726344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22005-e61d-4c5e-8220-b02a7fc75c4b None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.726696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22005-e61d-4c5e-8220-b02a7fc75c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.727118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22005-e61d-4c5e-8220-b02a7fc75c4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.727874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af22005-e61d-4c5e-8220-b02a7fc75c4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:19.729438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af22005-e61d-4c5e-8220-b02a7fc75c4b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:19.735587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:19.760261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 420/828] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:16:19 2022] POST /identity/v3/endpoints => generated 337 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:16:19.772083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc00da-cb92-4f2e-ac98-9d6781b2c497 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.772462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc00da-cb92-4f2e-ac98-9d6781b2c497 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.772741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc00da-cb92-4f2e-ac98-9d6781b2c497 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.774318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 421/829] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:19.780876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ae23-37b9-4f6b-8435-da19a66b4eae None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.781210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ae23-37b9-4f6b-8435-da19a66b4eae None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.781425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ae23-37b9-4f6b-8435-da19a66b4eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.818671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca5f94-4c92-4e61-9384-44edd463902f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:19.822242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f239ae23-37b9-4f6b-8435-da19a66b4eae None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:19.824889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f239ae23-37b9-4f6b-8435-da19a66b4eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:19.898441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:19.900376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:19.900866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:19.904534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:19.905556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:19.906083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:19.907689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:19.926506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 409/830] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:19.937319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:19.947489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 422/831] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:19.973361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ae23-37b9-4f6b-8435-da19a66b4eae None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:19.976273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:20.075010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2baaf-7576-4242-b45c-f4e3f130d1e2 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:20.103437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ae23-37b9-4f6b-8435-da19a66b4eae None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:20.122347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:20.123475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:20.123977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:20.124471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:20.134133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:20.136301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:20.143707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:20.146286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89003f70-490f-49d1-9690-da5f8fcc5034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:20.147034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89003f70-490f-49d1-9690-da5f8fcc5034 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:20.147624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89003f70-490f-49d1-9690-da5f8fcc5034 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:20.147998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89003f70-490f-49d1-9690-da5f8fcc5034 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:20.153058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:20.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:20.154899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:20.154899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:20.154899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:20.154899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:20.154899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:20.154899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:20.154899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:20.154899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:20.157589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 410/832] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:20.165653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:20.172019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89003f70-490f-49d1-9690-da5f8fcc5034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-89003f70-490f-49d1-9690-da5f8fcc5034 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:20.177350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:20.178320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:20.180781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 423/833] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:20.185716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89003f70-490f-49d1-9690-da5f8fcc5034 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:20.282339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:20.303250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01c289-04ff-4c50-b311-5d4291348154 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:20.310343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89003f70-490f-49d1-9690-da5f8fcc5034 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:20.344583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:20.345187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:20.345462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:20.345731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:20.347196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:20.347503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:20.349041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:20.357787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:20.359128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:20.359609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:20.360312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:20.361122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:20.361593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:20.363842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:20.368033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 411/834] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:20.369354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:20.374637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 424/835] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:16:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:20.379948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:20.433537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:20.489071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76895310-c5d0-4602-a09d-33260d4d0f9b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:20.507763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca7936-e696-440e-b4d8-32acd552c05a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:20.532869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:20.534424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:20.535002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:20.535297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:20.536181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=2e3819e78802469c83c09a1cdd865f58) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:20.536493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:20.538405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:20.551030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 412/836] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:20 2022] GET /identity/v3/role_assignments?user.id=2e3819e78802469c83c09a1cdd865f58&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=29fd031edd2b4f65aed8942853150f92 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:20.557695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:20.558352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:20.558630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:20.558902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:20.572090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:20.573510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:20.589578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:20.590596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:20.592307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:20.592307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:20.592307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:20.592307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 425/837] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:20.597246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:20.612419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:20.718854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769652c-ccb9-44e3-a0b0-e9f291cf0e9c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:20.756157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:20.757161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:20.757647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:20.758137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:20.758794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:20.759098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:20.760491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:20.775034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 413/838] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:20.780339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:20.881514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:20.888453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81c59e-748b-4630-8e3b-b8fb89bf241a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:20.924034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:20.924904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:20.925188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:20.925464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:20.933220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:20.934660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:20.946445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:20.947451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:20.948463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 426/839] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:16:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:20.953353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:21.026269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:21.060847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa853-12b6-4776-a5e0-b7531f767a8c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:21.097885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:21.098554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.098853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.099119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.099855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:21.100184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:21.101611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:21.121385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 414/840] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Aug 11 16:16:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:21.126695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:21.237974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c493dae-8667-4ec8-9a17-e32d018bb117 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:21.273721 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541ab6a-c0e2-4950-af30-aea8fd73255f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:21.274379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541ab6a-c0e2-4950-af30-aea8fd73255f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.274651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541ab6a-c0e2-4950-af30-aea8fd73255f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.274918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541ab6a-c0e2-4950-af30-aea8fd73255f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.301988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:21.346468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541ab6a-c0e2-4950-af30-aea8fd73255f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:21.347906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6541ab6a-c0e2-4950-af30-aea8fd73255f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:21.449334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 427/841] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:21 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:16:21.454661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:21.557533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92143f6-1214-4249-ba13-26a701569ce4 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.557846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92143f6-1214-4249-ba13-26a701569ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.558123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92143f6-1214-4249-ba13-26a701569ce4 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.559660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 428/842] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:21.566324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e781f51-8489-4159-aa98-223def80f444 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.566605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e781f51-8489-4159-aa98-223def80f444 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.566872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e781f51-8489-4159-aa98-223def80f444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.570875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4abff-00c9-42bd-b509-bbe7f9320873 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:21.585336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7e781f51-8489-4159-aa98-223def80f444 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:21.586614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7e781f51-8489-4159-aa98-223def80f444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:21.602196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:21.608673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:21.609584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.610070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.610603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.624055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:21.625809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:21.642066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:21.643685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:21.644724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:21.644724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:21.644724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:21.646870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 415/843] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:21.648505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:21.652296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:21.661727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 429/844] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:21.683704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e781f51-8489-4159-aa98-223def80f444 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:21.761764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da624998-01fb-413d-a2e6-db447175dc07 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:21.800447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e18e5-b5f1-464e-a191-bdc278a2e938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:21.801398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e18e5-b5f1-464e-a191-bdc278a2e938 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.801891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e18e5-b5f1-464e-a191-bdc278a2e938 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.802164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e18e5-b5f1-464e-a191-bdc278a2e938 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.802828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e18e5-b5f1-464e-a191-bdc278a2e938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:21.803132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e18e5-b5f1-464e-a191-bdc278a2e938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:21.804565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e18e5-b5f1-464e-a191-bdc278a2e938 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:21.811890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e781f51-8489-4159-aa98-223def80f444 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:21.819728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 416/845] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:21.821376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad75c7-8016-48a7-a915-ac89dbdcef35 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.822445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad75c7-8016-48a7-a915-ac89dbdcef35 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.822929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad75c7-8016-48a7-a915-ac89dbdcef35 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.824873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 417/846] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:21.826141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad75c7-8016-48a7-a915-ac89dbdcef35 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:21.850547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4f251-4699-4a27-97a9-02e72c444ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:21.851194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4f251-4699-4a27-97a9-02e72c444ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.851471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4f251-4699-4a27-97a9-02e72c444ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.851740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4f251-4699-4a27-97a9-02e72c444ba1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.852359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4f251-4699-4a27-97a9-02e72c444ba1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:21.853730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4f251-4699-4a27-97a9-02e72c444ba1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a3c4f251-4699-4a27-97a9-02e72c444ba1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:21.863192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:21.864254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:21.865168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:21.869393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 430/847] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:21.870603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2fc2-a66c-4466-95f9-c90af15fc4b8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.870894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2fc2-a66c-4466-95f9-c90af15fc4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.871173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2fc2-a66c-4466-95f9-c90af15fc4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.876449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:21.891138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d5ad2fc2-a66c-4466-95f9-c90af15fc4b8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:21.892361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d5ad2fc2-a66c-4466-95f9-c90af15fc4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:21.943787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad75c7-8016-48a7-a915-ac89dbdcef35 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:21.945744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:21.968914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 431/848] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:21.973668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad2fc2-a66c-4466-95f9-c90af15fc4b8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:21.980608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0638-883e-4307-baaf-c039135211ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:21.981235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0638-883e-4307-baaf-c039135211ca None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:21.981512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0638-883e-4307-baaf-c039135211ca None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:21.981780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d0638-883e-4307-baaf-c039135211ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:21.989110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d0638-883e-4307-baaf-c039135211ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:21.990652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d0638-883e-4307-baaf-c039135211ca None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-736d0638-883e-4307-baaf-c039135211ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:22.002106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:22.003815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 418/849] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:22.004787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0638-883e-4307-baaf-c039135211ca None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.099642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad2fc2-a66c-4466-95f9-c90af15fc4b8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:22.118965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:22.122506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d0638-883e-4307-baaf-c039135211ca None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:22.138405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:22.139016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.139292 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.139613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.140273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:22.140570 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:22.141978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:22.147129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 432/850] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:22.147942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.162592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4369ca-e610-40ff-bc62-f61726c46275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:22.163219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4369ca-e610-40ff-bc62-f61726c46275 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.163556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4369ca-e610-40ff-bc62-f61726c46275 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.163753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4369ca-e610-40ff-bc62-f61726c46275 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.175258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4369ca-e610-40ff-bc62-f61726c46275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-6f4369ca-e610-40ff-bc62-f61726c46275 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:22.181423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:22.182384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 419/851] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:22.183293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4369ca-e610-40ff-bc62-f61726c46275 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.203554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:22.260262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51b15c-0805-4995-b58e-ec3caa50eabd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:22.294694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4369ca-e610-40ff-bc62-f61726c46275 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:22.296887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:22.297695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.298136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.298558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.299465 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:22.299920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:22.301523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:22.314116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 433/852] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:22.315099 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.337479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:22.338173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.338633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.338969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.339672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:22.340042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:22.341591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:22.352426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 420/853] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:22.352426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.379533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:22.466606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d42c7-88f8-4f37-8a5e-bb3f2d3e6d19 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:22.474007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e06c23-de68-4563-aa18-736d4b32957a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:22.477941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:22.508765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:22.509431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.509781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.509963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.510744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=d7291ccb171e44f89dabc99980312c3b) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:22.511051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:22.512579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:22.520363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 421/854] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/role_assignments?user.id=d7291ccb171e44f89dabc99980312c3b&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:22.523402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa840e4d-0f7f-421b-9334-738738d09a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:22.524025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa840e4d-0f7f-421b-9334-738738d09a6e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.524354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa840e4d-0f7f-421b-9334-738738d09a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.524631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa840e4d-0f7f-421b-9334-738738d09a6e None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.525314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa840e4d-0f7f-421b-9334-738738d09a6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:22.525616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa840e4d-0f7f-421b-9334-738738d09a6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:22.527008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa840e4d-0f7f-421b-9334-738738d09a6e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:22.537584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 434/855] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:22.543251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.649472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:22.665152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c8f16-61af-4213-8285-92cb2d4e0e54 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:22.703782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb0fb6-53ec-4259-a47c-510a02763f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:22.704449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb0fb6-53ec-4259-a47c-510a02763f45 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.704724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb0fb6-53ec-4259-a47c-510a02763f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.704993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb0fb6-53ec-4259-a47c-510a02763f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.716232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb0fb6-53ec-4259-a47c-510a02763f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:22.717675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb0fb6-53ec-4259-a47c-510a02763f45 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:22.732166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da04e-a529-4fb1-b939-d71ce61e9110 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.732447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da04e-a529-4fb1-b939-d71ce61e9110 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.732721 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062da04e-a529-4fb1-b939-d71ce61e9110 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-04cb0fb6-53ec-4259-a47c-510a02763f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:22.732955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:22.733874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:22.735135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:22.735135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:22.735135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:22.735135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 422/857] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:22.735615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 435/857] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:22.740397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cfce0-282d-4054-a2c9-bc2c8b700460 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.740686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cfce0-282d-4054-a2c9-bc2c8b700460 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.740960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cfce0-282d-4054-a2c9-bc2c8b700460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.741384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062da04e-a529-4fb1-b939-d71ce61e9110 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.763306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f14cfce0-282d-4054-a2c9-bc2c8b700460 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:22.764559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f14cfce0-282d-4054-a2c9-bc2c8b700460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:22.854130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 423/858] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:22.870862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062da04e-a529-4fb1-b939-d71ce61e9110 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:22.877047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cfce0-282d-4054-a2c9-bc2c8b700460 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.909131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:22.909745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:22.910026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:22.910298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:22.910950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:22.911251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:22.912651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:22.932145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 436/859] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:22.936090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:22.967883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:22.976840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:23.003329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cfce0-282d-4054-a2c9-bc2c8b700460 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:23.046799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:23.047812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:23.048336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:23.048809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:23.049809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:23.051395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:23.064908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:23.066045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:23.067466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:23.068805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef48ba-c7ff-4e04-aed1-2249ee8dc871 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:23.070511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 424/860] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:23.080166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:23.107584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:23.108181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:23.108460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:23.108726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:23.117043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:23.118810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:23.130722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:23.131794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 11 16:16:23.132823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:23.136295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 437/861] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:23.137993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:23.186834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149d63-f666-4db3-aa71-b8230f5baff2 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:23.222867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:23.223564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:23.223841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:23.224123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:23.224772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:23.225069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:23.226631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:23.227779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:23.230834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:23.237316 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 425/862] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:23.238242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:23.248977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2682df-9c56-43af-a625-f20a9c00722c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:23.286469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:23.287352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:23.287785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:23.288224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:23.289195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:23.289778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:23.291322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:23.315866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 438/863] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Aug 11 16:16:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:23.317397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:23.347696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c6855-d9c7-44c2-8bcc-db50a4af6521 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:23.385477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:23.386087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:23.386362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:23.386628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:23.387516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:23.387814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:23.389943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:23.400265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 426/864] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:23 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:23.433090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1889fda3-5ac0-419c-98dc-fe084afd4fb7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:23.473876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10422232-ecc7-4ed3-95d6-4323a034ff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:23.474898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422232-ecc7-4ed3-95d6-4323a034ff65 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:23.475372 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422232-ecc7-4ed3-95d6-4323a034ff65 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:23.475818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10422232-ecc7-4ed3-95d6-4323a034ff65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:23.493078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:23.554535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10422232-ecc7-4ed3-95d6-4323a034ff65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:23.556607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10422232-ecc7-4ed3-95d6-4323a034ff65 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:23.671812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 439/865] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 16:16:23.673243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:23.737748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:23.777992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b246357-2ed2-4501-9186-fa5387b8e64b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:23.820286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:23.821217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:23.821699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:23.822141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:23.835295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:23.837015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:23.852457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:23.853449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:23.854345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:23.854345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:23.854345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:23.856140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 427/866] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:23.861685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10422232-ecc7-4ed3-95d6-4323a034ff65 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:23.910714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:23.968039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10422232-ecc7-4ed3-95d6-4323a034ff65 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:24.008496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:24.009031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.009305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.009572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.010219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:24.010520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:24.011873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:24.029429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 440/867] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:24.030756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:24.140084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf61da3-0fbe-473d-8371-2632d0f00c8f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:24.154966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:24.177788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:24.178648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.179126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.179564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.186675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:24.188353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:24.200588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:24.201700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:24.202802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:24.202802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:24.202802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:24.202802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:24.202802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:24.202802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:24.202802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:24.202802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:24.204897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 428/868] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:24.216245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:24.321967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc192f2-5fc6-4f9e-80a4-8b1963065918 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:24.325118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:24.358344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ee347-5ec1-4b45-9852-f22b257b7b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:24.358952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ee347-5ec1-4b45-9852-f22b257b7b66 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.359275 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ee347-5ec1-4b45-9852-f22b257b7b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.359606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ee347-5ec1-4b45-9852-f22b257b7b66 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.360376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ee347-5ec1-4b45-9852-f22b257b7b66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:24.360675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ee347-5ec1-4b45-9852-f22b257b7b66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:24.362776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ee347-5ec1-4b45-9852-f22b257b7b66 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:24.375238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 441/869] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:24.382406 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:24.490390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea82ea3-2e64-435e-a13c-a5c67d11d584 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:24.519478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e414f-09c7-492f-9a3e-04e529548f66 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.519817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e414f-09c7-492f-9a3e-04e529548f66 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.520019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e414f-09c7-492f-9a3e-04e529548f66 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.521496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 442/870] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:24.526582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204e96f-b536-47a9-87a3-d7affa2436e3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.527108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204e96f-b536-47a9-87a3-d7affa2436e3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.527591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204e96f-b536-47a9-87a3-d7affa2436e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.528789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:24.529385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.529658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.529924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.530646 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=256cc27b87684392a22f36429b960644) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:24.530945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:24.532379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:24.540049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 429/871] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:24 2022] GET /identity/v3/role_assignments?user.id=256cc27b87684392a22f36429b960644&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=29fd031edd2b4f65aed8942853150f92 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:24.564131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4204e96f-b536-47a9-87a3-d7affa2436e3 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:24.564131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4204e96f-b536-47a9-87a3-d7affa2436e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:24.638770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 443/872] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:24.662258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:24.708556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:24.741114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e5ab2-26e2-4086-9a62-d57269ba51fd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.741424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e5ab2-26e2-4086-9a62-d57269ba51fd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.741664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e5ab2-26e2-4086-9a62-d57269ba51fd None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.743292 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 444/873] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:24.748917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262fae7-18d5-42e4-8960-e1bd72b95a2e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.749253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262fae7-18d5-42e4-8960-e1bd72b95a2e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.749515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262fae7-18d5-42e4-8960-e1bd72b95a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.772224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a262fae7-18d5-42e4-8960-e1bd72b95a2e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:24.773480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a262fae7-18d5-42e4-8960-e1bd72b95a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:24.788375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4183a-1a22-4625-b119-a9d4dfc7f4d8 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:24.830541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d42c2-e77d-4d82-8ad6-eaed2345555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:24.830541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d42c2-e77d-4d82-8ad6-eaed2345555b None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:24.830541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d42c2-e77d-4d82-8ad6-eaed2345555b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:24.830541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d42c2-e77d-4d82-8ad6-eaed2345555b None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:24.831282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d42c2-e77d-4d82-8ad6-eaed2345555b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:24.832746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d42c2-e77d-4d82-8ad6-eaed2345555b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:24.852877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 430/874] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:16:24 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:24.875935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 445/875] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:24.899303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d42c2-e77d-4d82-8ad6-eaed2345555b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:25.013778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:25.018401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d42c2-e77d-4d82-8ad6-eaed2345555b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:25.055321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cff65-d234-4fb9-be75-369a55157213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:25.056025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cff65-d234-4fb9-be75-369a55157213 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.056333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cff65-d234-4fb9-be75-369a55157213 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.056607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cff65-d234-4fb9-be75-369a55157213 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.068528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539cff65-d234-4fb9-be75-369a55157213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-539cff65-d234-4fb9-be75-369a55157213 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:25.073277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:25.074234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:25.075094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 431/876] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Aug 11 16:16:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:25.083951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262fae7-18d5-42e4-8960-e1bd72b95a2e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:25.106578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:25.195652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262fae7-18d5-42e4-8960-e1bd72b95a2e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:25.231815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:25.232458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.232738 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.233011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.233681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:25.233983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:25.235342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:25.244072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 446/877] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Aug 11 16:16:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:25.253590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cff65-d234-4fb9-be75-369a55157213 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:25.361702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539cff65-d234-4fb9-be75-369a55157213 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:25.397444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:25.398085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.398668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.398946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.410469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:25.411977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:25.427266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:25.428268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:25.429193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:25.429193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:25.429193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:25.429193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 432/878] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:25.435147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:25.528450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:25.548243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a9e8-6341-4705-b5cb-5ce688659442 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:25.584877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e1204-6ce7-4c09-bb69-c05f542510a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:25.585548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1204-6ce7-4c09-bb69-c05f542510a0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.585832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1204-6ce7-4c09-bb69-c05f542510a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.586102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1204-6ce7-4c09-bb69-c05f542510a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.586748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e1204-6ce7-4c09-bb69-c05f542510a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:25.587046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e1204-6ce7-4c09-bb69-c05f542510a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:25.588444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e1204-6ce7-4c09-bb69-c05f542510a0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:25.601238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 447/879] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:25.606723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:25.620917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:25.663487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475c03-d7d8-4c39-a376-1a6e6a38b329 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.664217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475c03-d7d8-4c39-a376-1a6e6a38b329 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.664803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475c03-d7d8-4c39-a376-1a6e6a38b329 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.666704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 448/880] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:25.673164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43bd8c-cbbe-441c-afc5-582d958eeef7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.673665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43bd8c-cbbe-441c-afc5-582d958eeef7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.674573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43bd8c-cbbe-441c-afc5-582d958eeef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.694769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2b43bd8c-cbbe-441c-afc5-582d958eeef7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:25.696320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2b43bd8c-cbbe-441c-afc5-582d958eeef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:25.726017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258cb2b-8c2f-4af7-ae6e-3d16cfa437b7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:25.762173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:25.762898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.763177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.763584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.772430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 449/881] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:25 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:25.772888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:25.774352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:25.791435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:25.792467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:25.793457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:25.795665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 433/882] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:16:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:25.801187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43bd8c-cbbe-441c-afc5-582d958eeef7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:25.803186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:25.887720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:25.891995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:25.929516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43bd8c-cbbe-441c-afc5-582d958eeef7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:25.930688 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbfd09-6e95-4e1e-b997-1674c3ec42e6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:25.969695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:25.970913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.971794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.972348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.973209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98799938-7282-4906-a0e7-c46586fffa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:25.973602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:25.973811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799938-7282-4906-a0e7-c46586fffa29 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:25.974087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799938-7282-4906-a0e7-c46586fffa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:25.974354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98799938-7282-4906-a0e7-c46586fffa29 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:25.974705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:25.975000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98799938-7282-4906-a0e7-c46586fffa29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:25.976399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98799938-7282-4906-a0e7-c46586fffa29 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:25.976760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:26.000507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 450/883] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:16:25 2022] POST /identity/v3/services => generated 269 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:26.011036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 434/884] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Aug 11 16:16:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:26.019742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98799938-7282-4906-a0e7-c46586fffa29 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:26.136720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98799938-7282-4906-a0e7-c46586fffa29 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:26.169095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:26.175462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73322175-6767-44da-8b2a-424ecdf3f965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:26.176646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73322175-6767-44da-8b2a-424ecdf3f965 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:26.177311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73322175-6767-44da-8b2a-424ecdf3f965 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:26.177586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73322175-6767-44da-8b2a-424ecdf3f965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:26.249628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73322175-6767-44da-8b2a-424ecdf3f965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:26.251071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73322175-6767-44da-8b2a-424ecdf3f965 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:26.357062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 451/885] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:26 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:16:26.362927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:26.468163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7bae5-41d1-4c5f-8327-ec1f19f0ae87 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:26.497548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:26.506010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730e655-2d71-43ff-9bd8-0b1ec2977000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:26.506636 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730e655-2d71-43ff-9bd8-0b1ec2977000 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:26.506929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730e655-2d71-43ff-9bd8-0b1ec2977000 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:26.507170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730e655-2d71-43ff-9bd8-0b1ec2977000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:26.517598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730e655-2d71-43ff-9bd8-0b1ec2977000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:26.519268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730e655-2d71-43ff-9bd8-0b1ec2977000 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-d730e655-2d71-43ff-9bd8-0b1ec2977000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:26.534775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:26.535758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:26.536701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:26.536701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:26.536701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:26.540205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 435/886] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:26.541690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73322175-6767-44da-8b2a-424ecdf3f965 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:26.572321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:26.650663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73322175-6767-44da-8b2a-424ecdf3f965 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:26.687648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:26.688934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:26.689423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:26.690513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:26.691370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:26.691895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:26.693486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:26.698422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1c50-4029-4a75-bf28-a0a02d8e628e None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:26.698717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1c50-4029-4a75-bf28-a0a02d8e628e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:26.698981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a1c50-4029-4a75-bf28-a0a02d8e628e None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:26.700600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 436/887] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:26.705453 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286e4a-83bf-47ca-bcf4-e80c5b34478c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:26.705737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286e4a-83bf-47ca-bcf4-e80c5b34478c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:26.706014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f286e4a-83bf-47ca-bcf4-e80c5b34478c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:26.707922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 452/888] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:26.716961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:26.724400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-6f286e4a-83bf-47ca-bcf4-e80c5b34478c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:26.725589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6f286e4a-83bf-47ca-bcf4-e80c5b34478c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:26.804445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 437/889] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:26.825912 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:26.834421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f286e4a-83bf-47ca-bcf4-e80c5b34478c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:26.840061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcd6bf-21e5-4f8a-80f7-255c22445e55 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:26.876487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:26.877176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:26.877969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:26.878239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:26.886355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:26.888631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:26.900892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:26.901824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:26.902750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:26.902750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:26.902750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:26.902750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:26.902750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:26.902750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:26.902750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:26.902750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:26.903209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 453/890] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:26.909239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:26.957641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f286e4a-83bf-47ca-bcf4-e80c5b34478c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:26.971939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:27.004180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:27.005082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.005432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.005776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.007281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:27.008765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:27.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:27.020829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:27.021851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:27.024247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 438/891] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:27.028141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:27.047414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5eaf93-b1d5-43be-87eb-0c72c61ef245 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:27.082764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35abec3-754b-4506-af8c-894e78806dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:27.083362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35abec3-754b-4506-af8c-894e78806dcb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.083633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35abec3-754b-4506-af8c-894e78806dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.083904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35abec3-754b-4506-af8c-894e78806dcb None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.084633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35abec3-754b-4506-af8c-894e78806dcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:27.084933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35abec3-754b-4506-af8c-894e78806dcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:27.086334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35abec3-754b-4506-af8c-894e78806dcb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:27.104646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 454/892] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:27.105792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5eede-2b2a-44e5-b1ab-29a621430722 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.106073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5eede-2b2a-44e5-b1ab-29a621430722 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.106340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5eede-2b2a-44e5-b1ab-29a621430722 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.107722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 455/893] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:27.108277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5eede-2b2a-44e5-b1ab-29a621430722 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:27.119224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:27.145761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20887f99-8212-4178-8e42-67a4fc6083c6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:27.190262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76134c-2de1-4c71-9be3-257065149662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:27.190851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76134c-2de1-4c71-9be3-257065149662 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.191074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76134c-2de1-4c71-9be3-257065149662 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.191346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76134c-2de1-4c71-9be3-257065149662 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.192186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76134c-2de1-4c71-9be3-257065149662 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:27.192875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76134c-2de1-4c71-9be3-257065149662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:27.194518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76134c-2de1-4c71-9be3-257065149662 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:27.200637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 439/894] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:27.206389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592b04-a0a5-4c59-907d-e729d6198d8e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.206670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592b04-a0a5-4c59-907d-e729d6198d8e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.206937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592b04-a0a5-4c59-907d-e729d6198d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.216005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5eede-2b2a-44e5-b1ab-29a621430722 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:27.225482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-2e592b04-a0a5-4c59-907d-e729d6198d8e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:27.226688 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2e592b04-a0a5-4c59-907d-e729d6198d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:27.253975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:27.254742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.255024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.255297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.256026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] RBAC: Adding query filter params (role.id=34515f0849514506bafe075b65e3aa5e, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=d7291ccb171e44f89dabc99980312c3b) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:27.256371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:27.256751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:27.257805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:27.264084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 456/895] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:27 2022] GET /identity/v3/role_assignments?user.id=d7291ccb171e44f89dabc99980312c3b&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=34515f0849514506bafe075b65e3aa5e => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:27.264946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:27.319021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 440/896] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:27.341176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592b04-a0a5-4c59-907d-e729d6198d8e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:27.365974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:27.405199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8995e-442c-4e5e-af62-b2bbb69dac5a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:27.445765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:27.446722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.447209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.447622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.448463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:27.448901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:27.450268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:27.464593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 457/897] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:27.478704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592b04-a0a5-4c59-907d-e729d6198d8e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:27.520214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:27.521308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.521791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.522363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.523166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:27.524933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:27.535774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:27.536876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:27.537725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:27.545555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 441/898] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:27.547305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:27.661790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c3fd2-d2a0-4406-80d4-03ea346b0bed None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:27.696656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:27.697605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.698108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.698577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.699404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:27.699917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:27.701560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:27.708454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:27.712225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 458/899] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:27.713617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:27.749383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:27.823097 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e47ff-6fed-428c-843f-71727ab28199 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:27.858866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:27.859470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:27.859748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:27.860018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:27.860686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:27.860999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:27.862986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:27.872214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 442/900] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:27.874328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:27.978686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504c0af-c6de-43cf-a142-478aa9f56e7c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:28.013604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26496487-9346-403e-88bf-2d76fe2775d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:28.014268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496487-9346-403e-88bf-2d76fe2775d9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:28.014549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496487-9346-403e-88bf-2d76fe2775d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:28.014874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26496487-9346-403e-88bf-2d76fe2775d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:28.015579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496487-9346-403e-88bf-2d76fe2775d9 None admin] RBAC: Adding query filter params (interface=public, service_id=f6f533900b704e6586697df2ad04728c, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:28.015882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496487-9346-403e-88bf-2d76fe2775d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:28.017331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26496487-9346-403e-88bf-2d76fe2775d9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:28.027741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 459/901] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Aug 11 16:16:27 2022] GET /identity/v3/endpoints?service_id=f6f533900b704e6586697df2ad04728c&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:28.029121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:28.129563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:28.135360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c609ee-f011-4b88-b8a3-179eb42c3132 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:28.170601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcd866-336b-412b-bebc-4219308a7bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:28.171207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcd866-336b-412b-bebc-4219308a7bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:28.171483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcd866-336b-412b-bebc-4219308a7bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:28.171750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcd866-336b-412b-bebc-4219308a7bf4 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:28.172386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcd866-336b-412b-bebc-4219308a7bf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:28.173805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dcd866-336b-412b-bebc-4219308a7bf4 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:28.184241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 443/902] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:16:28 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:16:28.202052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe664-c718-4511-b2e7-bd6eb0d6d323 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:28.202378 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe664-c718-4511-b2e7-bd6eb0d6d323 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:28.202594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfe664-c718-4511-b2e7-bd6eb0d6d323 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:28.204322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 460/903] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:28.210331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775fd4-4a76-453b-93af-6cbd852ba2f0 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:28.210650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775fd4-4a76-453b-93af-6cbd852ba2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:28.210881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a775fd4-4a76-453b-93af-6cbd852ba2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:28.230247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4a775fd4-4a76-453b-93af-6cbd852ba2f0 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:28.231436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4a775fd4-4a76-453b-93af-6cbd852ba2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:28.310986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 444/904] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:28.334835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfe664-c718-4511-b2e7-bd6eb0d6d323 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:28.403456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:28.447668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfe664-c718-4511-b2e7-bd6eb0d6d323 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:28.483375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:28.484064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:28.484435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:28.484773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:28.485420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:28.486969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:28.496141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:28.497272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:28.498388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:28.499415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 461/905] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Aug 11 16:16:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:28.505199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a775fd4-4a76-453b-93af-6cbd852ba2f0 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:28.602446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:28.613187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a775fd4-4a76-453b-93af-6cbd852ba2f0 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:28.656512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:28.657225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:28.657574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:28.657915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:28.659211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:28.659611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:28.661308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:28.666779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 445/906] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:28.673164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:28.775214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866d057-d0a6-4245-bdc2-4fb647636d86 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:28.803370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:28.815337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:28.816167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:28.816612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:28.817022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:28.817826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:28.818263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:28.819749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:28.825300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 462/907] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:28.831074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:28.935537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9d5b9-fb60-4f67-bc43-18917f6b7988 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:28.974933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1aa338-5bb2-4d91-b3a4-23f22c4609ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:28.975556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aa338-5bb2-4d91-b3a4-23f22c4609ff None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:28.975833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aa338-5bb2-4d91-b3a4-23f22c4609ff None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:28.976118 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aa338-5bb2-4d91-b3a4-23f22c4609ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:28.976835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1aa338-5bb2-4d91-b3a4-23f22c4609ff None admin] RBAC: Adding query filter params (interface=public, service_id=0a19b3c6c2284a249a1f50f58d03a8e0, region_id=RegionOne) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:28.977140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1aa338-5bb2-4d91-b3a4-23f22c4609ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:28.978548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1aa338-5bb2-4d91-b3a4-23f22c4609ff None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:28.984111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 446/908] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Aug 11 16:16:28 2022] GET /identity/v3/endpoints?service_id=0a19b3c6c2284a249a1f50f58d03a8e0&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:28.990290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:29.098991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cd815-cd3d-438c-b7e1-3c3c16f6b103 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:29.134032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d454b01-11b3-4fdc-94af-2a125d321608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:29.134646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454b01-11b3-4fdc-94af-2a125d321608 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:29.134926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454b01-11b3-4fdc-94af-2a125d321608 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:29.135200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454b01-11b3-4fdc-94af-2a125d321608 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:29.135862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d454b01-11b3-4fdc-94af-2a125d321608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:29.137258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d454b01-11b3-4fdc-94af-2a125d321608 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:29.144295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 463/909] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:16:28 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:16:29.483571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e662181-43b8-4384-8592-da54d7baeed5 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:29.483990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e662181-43b8-4384-8592-da54d7baeed5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:29.484299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e662181-43b8-4384-8592-da54d7baeed5 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:29.485908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 447/910] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:29.492014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ee04d-c5b2-4bab-b95b-cc53ed22e7d8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:29.492328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ee04d-c5b2-4bab-b95b-cc53ed22e7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:29.492604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ee04d-c5b2-4bab-b95b-cc53ed22e7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:29.512036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-451ee04d-c5b2-4bab-b95b-cc53ed22e7d8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:29.513257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-451ee04d-c5b2-4bab-b95b-cc53ed22e7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:29.566300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:29.588210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 464/911] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:29.600405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf633a-fd99-42e6-8f85-fa56439c937e None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:29.600692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf633a-fd99-42e6-8f85-fa56439c937e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:29.600967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf633a-fd99-42e6-8f85-fa56439c937e None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:29.602306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 448/912] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:29.607868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf633a-fd99-42e6-8f85-fa56439c937e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:29.608156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68767601-d419-406b-838e-0c0596b1df2e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:29.608449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68767601-d419-406b-838e-0c0596b1df2e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:29.608725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68767601-d419-406b-838e-0c0596b1df2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:29.628459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-68767601-d419-406b-838e-0c0596b1df2e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:29.629635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-68767601-d419-406b-838e-0c0596b1df2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:29.632520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:29.700977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 465/913] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:29.726050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68767601-d419-406b-838e-0c0596b1df2e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:29.744118 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf633a-fd99-42e6-8f85-fa56439c937e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:29.781739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7ab1c-3573-413e-b09d-e7e3c5382289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:29.782364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ab1c-3573-413e-b09d-e7e3c5382289 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:29.782640 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ab1c-3573-413e-b09d-e7e3c5382289 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:29.782906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ab1c-3573-413e-b09d-e7e3c5382289 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:29.795396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7ab1c-3573-413e-b09d-e7e3c5382289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-a3a7ab1c-3573-413e-b09d-e7e3c5382289 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:29.801267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:29.802206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:29.807451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 449/914] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Aug 11 16:16:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:29.808603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7ab1c-3573-413e-b09d-e7e3c5382289 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:29.831023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68767601-d419-406b-838e-0c0596b1df2e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:29.848369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:29.868376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5728f-4ae3-4f26-89c1-c1133431b090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:29.868993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5728f-4ae3-4f26-89c1-c1133431b090 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:29.869578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5728f-4ae3-4f26-89c1-c1133431b090 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:29.869864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5728f-4ae3-4f26-89c1-c1133431b090 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:29.870523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5728f-4ae3-4f26-89c1-c1133431b090 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:29.871906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5728f-4ae3-4f26-89c1-c1133431b090 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:29.890620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 466/915] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:16:29 2022] POST /identity/v3/services => generated 241 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:16:29.916207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:29.943990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7ab1c-3573-413e-b09d-e7e3c5382289 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:29.990559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:29.991187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:29.991467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:29.991788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:29.992624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:29.992931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:29.994510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:30.005418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 450/916] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Aug 11 16:16:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:30.011005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5728f-4ae3-4f26-89c1-c1133431b090 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:30.137008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5728f-4ae3-4f26-89c1-c1133431b090 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:30.173706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0dde5-82f0-43fe-bcfe-593381309063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:30.174504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0dde5-82f0-43fe-bcfe-593381309063 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.174785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0dde5-82f0-43fe-bcfe-593381309063 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.175059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0dde5-82f0-43fe-bcfe-593381309063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.186357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0dde5-82f0-43fe-bcfe-593381309063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:30.187801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0dde5-82f0-43fe-bcfe-593381309063 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-49c0dde5-82f0-43fe-bcfe-593381309063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:30.203287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:30.204311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:30.205853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:30.205853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:30.205853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:30.205853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 467/917] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:30.212276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:30.288661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a2e4f-c25d-48ca-a62f-060205164f40 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.288957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a2e4f-c25d-48ca-a62f-060205164f40 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.289135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a2e4f-c25d-48ca-a62f-060205164f40 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.290762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 468/918] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:30.295935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb61ee5-4ff9-4338-b47d-d8f2c25c271a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.296253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb61ee5-4ff9-4338-b47d-d8f2c25c271a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.296591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb61ee5-4ff9-4338-b47d-d8f2c25c271a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.316788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9eb61ee5-4ff9-4338-b47d-d8f2c25c271a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:30.318043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9eb61ee5-4ff9-4338-b47d-d8f2c25c271a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:30.320733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd341d4a-50fc-481b-a81d-1728400c0ef1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:30.343610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:30.360201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:30.360812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.361087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.361882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.362542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:30.362867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:30.364229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:30.379732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 451/919] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:30.380500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:30.388145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:30.408022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 469/920] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:30.425194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb61ee5-4ff9-4338-b47d-d8f2c25c271a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:30.512860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489afd0-c7ed-4e1f-b122-53c6d3f4d749 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:30.550320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:30.550972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.551253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.551524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.554172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb61ee5-4ff9-4338-b47d-d8f2c25c271a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:30.559107 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:30.560788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:30.573792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:30.574764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:30.575730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 452/921] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:16:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:30.581857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:30.601429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:30.602281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.602563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.602836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.603852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:30.605611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:30.617595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:30.618773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 11 16:16:30.619703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:30.624691 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 470/922] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:30.626314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:30.662749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:30.666173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:30.723999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ede805-25bc-47e3-8b6d-70bb3d176145 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:30.761363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:30.762023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.762352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.762571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.763302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:30.763602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:30.765015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a9814-43a9-4949-b5a0-a72c277f8319 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:30.765464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:30.788807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 453/923] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Aug 11 16:16:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:30.789887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:30.814980 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:30.815816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.816526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.816945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.817959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:30.818403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:30.820632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:30.827619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 471/924] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:30.835359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:30.921581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78b566-0055-491b-9f60-cac2993971ef None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:30.926433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:30.942935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:30.944631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f426-b362-4126-9b1f-d029ce272649 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:30.968530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:30.969421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.969993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.970406 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.984155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:30.986033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:30.989399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:30.990584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:30.991004 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:30.991436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:30.992248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:30.992688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:30.994229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:31.001287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 472/925] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Aug 11 16:16:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:31.008237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:31.009214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:31.010139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:31.010139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:31.010139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:31.011505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:31.013978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 454/926] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:31.015126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:31.141307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb75f28-98fb-4c38-b4b3-589abc517753 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:31.147885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588bb4f-d5ed-405e-877e-af614f61f6f0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:31.177696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c2991-a8ef-4c8b-837b-c314309a558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:31.178311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c2991-a8ef-4c8b-837b-c314309a558c None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.178636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c2991-a8ef-4c8b-837b-c314309a558c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.178917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c2991-a8ef-4c8b-837b-c314309a558c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.179651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c2991-a8ef-4c8b-837b-c314309a558c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:31.181047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c2991-a8ef-4c8b-837b-c314309a558c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:31.191296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21d232-599e-49ae-a876-8e1c1d23e9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:31.192374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21d232-599e-49ae-a876-8e1c1d23e9c8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.192788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21d232-599e-49ae-a876-8e1c1d23e9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.193076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21d232-599e-49ae-a876-8e1c1d23e9c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.193705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21d232-599e-49ae-a876-8e1c1d23e9c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:31.194027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21d232-599e-49ae-a876-8e1c1d23e9c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:31.195367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21d232-599e-49ae-a876-8e1c1d23e9c8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:31.205225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 473/927] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:16:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:16:31.223777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 455/928] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:31.225357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c2991-a8ef-4c8b-837b-c314309a558c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:31.236157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:31.269934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f4f7-d63b-4d0d-92c9-b82b30df224b None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.270286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f4f7-d63b-4d0d-92c9-b82b30df224b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.270663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116f4f7-d63b-4d0d-92c9-b82b30df224b None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.276007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 456/929] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:31.277613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d686e-a9c3-4262-9546-a8843602e5e2 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.277896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d686e-a9c3-4262-9546-a8843602e5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.278169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d686e-a9c3-4262-9546-a8843602e5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.302580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-642d686e-a9c3-4262-9546-a8843602e5e2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:31.304419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-642d686e-a9c3-4262-9546-a8843602e5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:31.335844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:31.368807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c2991-a8ef-4c8b-837b-c314309a558c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:31.404287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 457/930] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:31.413341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:31.413977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.414251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.414517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.421895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:31.423323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:31.428206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d686e-a9c3-4262-9546-a8843602e5e2 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:31.438981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:31.439991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:31.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 474/931] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:31.453097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:31.525237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:31.572621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7313e59-5221-48fe-b88c-95be0b97267a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:31.573631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d686e-a9c3-4262-9546-a8843602e5e2 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:31.611994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:31.612683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.612963 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.613269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.613969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:31.614267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:31.615736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:31.621656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:31.622611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.623144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.623626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.625059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:31.628429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:31.635199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 475/932] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:31.636178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:31.638872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:31.639894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 11 16:16:31.640920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:31.642694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 458/933] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Aug 11 16:16:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:31.649068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:31.674286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:31.761907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04261d-b88a-446f-8be2-90929c5d9afd None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:31.769493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32461d5b-650d-4faa-b35e-05083e733410 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:31.801091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:31.811052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:31.811981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.812294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.812565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.813251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:31.813541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:31.815747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:31.817971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9e54a-3df5-4af8-b675-180303860a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:31.818934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9e54a-3df5-4af8-b675-180303860a6c None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.819417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9e54a-3df5-4af8-b675-180303860a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.819911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9e54a-3df5-4af8-b675-180303860a6c None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/d7291ccb171e44f89dabc99980312c3b/roles/34515f0849514506bafe075b65e3aa5e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.821230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 459/934] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:31.822706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9e54a-3df5-4af8-b675-180303860a6c None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=d7291ccb171e44f89dabc99980312c3b, role_id=34515f0849514506bafe075b65e3aa5e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:31.825361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9e54a-3df5-4af8-b675-180303860a6c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:31.826528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:31.842218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 476/935] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:16:31 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/d7291ccb171e44f89dabc99980312c3b/roles/34515f0849514506bafe075b65e3aa5e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:16:31.935068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:31.954889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b6ff2c-50b1-41d6-9208-192092155b06 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:31.995445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4de28e-01cb-4b95-b4a8-b8dbe805c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:31.996062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4de28e-01cb-4b95-b4a8-b8dbe805c081 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:31.996364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4de28e-01cb-4b95-b4a8-b8dbe805c081 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:31.996638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4de28e-01cb-4b95-b4a8-b8dbe805c081 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:31.997339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4de28e-01cb-4b95-b4a8-b8dbe805c081 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:31.997645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4de28e-01cb-4b95-b4a8-b8dbe805c081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:31.999179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4de28e-01cb-4b95-b4a8-b8dbe805c081 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:32.005009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 460/936] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:32.010707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9e54a-3df5-4af8-b675-180303860a6c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:32.061262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cc916-db9c-42bd-b2e6-8b4d7988554d None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:32.061699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cc916-db9c-42bd-b2e6-8b4d7988554d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:32.062075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cc916-db9c-42bd-b2e6-8b4d7988554d None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:32.064033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 461/937] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:32.069515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29da2ff-1d92-461d-b889-e7caff054e79 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:32.069885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29da2ff-1d92-461d-b889-e7caff054e79 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:32.070255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29da2ff-1d92-461d-b889-e7caff054e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:32.089566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f29da2ff-1d92-461d-b889-e7caff054e79 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:32.090911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f29da2ff-1d92-461d-b889-e7caff054e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:32.140696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9e54a-3df5-4af8-b675-180303860a6c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:32.169656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 462/938] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:32.178239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e48316-8703-46ef-aed2-d166575206ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:32.179017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e48316-8703-46ef-aed2-d166575206ae None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:32.179453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e48316-8703-46ef-aed2-d166575206ae None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:32.179877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e48316-8703-46ef-aed2-d166575206ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:32.180710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e48316-8703-46ef-aed2-d166575206ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:32.182220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e48316-8703-46ef-aed2-d166575206ae None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:32.194050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29da2ff-1d92-461d-b889-e7caff054e79 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:32.208624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 477/939] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:16:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:32.292251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:32.318873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29da2ff-1d92-461d-b889-e7caff054e79 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:32.368111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:32.368765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:32.369043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:32.369315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:32.369907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:32.371358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:32.381273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:32.382346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:32.383199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 463/940] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:32.388859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e48316-8703-46ef-aed2-d166575206ae None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:32.434442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:32.503167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e48316-8703-46ef-aed2-d166575206ae None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:32.539087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:32.539748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:32.540032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:32.540330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:32.540961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:32.541314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:32.542678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:32.548052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 478/941] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:32.554410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:32.669916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac633f-3e8e-407e-8f05-f6ad8cbde5dc None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:32.706379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:32.706992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:32.707270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:32.707545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:32.708224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:32.708719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:32.710196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:32.715588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 464/942] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:32.721493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:32.827298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27a946-e4c7-4db3-a679-248e054818c4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:32.828886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:32.863197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8462f-18c2-4b31-a063-8a06c0732601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:32.863808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8462f-18c2-4b31-a063-8a06c0732601 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:32.864164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8462f-18c2-4b31-a063-8a06c0732601 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:32.864440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8462f-18c2-4b31-a063-8a06c0732601 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:32.865221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8462f-18c2-4b31-a063-8a06c0732601 None admin] RBAC: Adding query filter params (interface=public, service_id=8b8a1c4dd644495badf6877ad52521cc, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:32.865533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8462f-18c2-4b31-a063-8a06c0732601 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:32.866970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e8462f-18c2-4b31-a063-8a06c0732601 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:32.872543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 479/943] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Aug 11 16:16:32 2022] GET /identity/v3/endpoints?service_id=8b8a1c4dd644495badf6877ad52521cc&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:32.878706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:32.881633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:32.984622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31889edd-7faa-4009-b940-16119bbb7338 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:33.026140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b9d2e-d546-40de-9d93-103221c4617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:33.026780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b9d2e-d546-40de-9d93-103221c4617f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:33.027060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b9d2e-d546-40de-9d93-103221c4617f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:33.027331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b9d2e-d546-40de-9d93-103221c4617f None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:33.027960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b9d2e-d546-40de-9d93-103221c4617f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:33.029405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b9d2e-d546-40de-9d93-103221c4617f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:33.037295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 465/944] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:16:32 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:16:33.404751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a82cb-0b7b-4435-a421-a40241c3c59a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:33.405437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a82cb-0b7b-4435-a421-a40241c3c59a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:33.405949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a82cb-0b7b-4435-a421-a40241c3c59a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:33.407736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 480/945] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:33.413090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791abfe-3cc6-4adc-a508-3d033cca6c6f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:33.413376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791abfe-3cc6-4adc-a508-3d033cca6c6f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:33.413651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791abfe-3cc6-4adc-a508-3d033cca6c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:33.432796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1791abfe-3cc6-4adc-a508-3d033cca6c6f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:33.434030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1791abfe-3cc6-4adc-a508-3d033cca6c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:33.500871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:33.511257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 466/946] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:33.534244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a82cb-0b7b-4435-a421-a40241c3c59a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:33.640273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a82cb-0b7b-4435-a421-a40241c3c59a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:33.681125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:33.681659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:33.681941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:33.682215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:33.682870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:33.684319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:33.686933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING keystone.server.flask.common [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 11 16:16:33.707207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.sql.core [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 16:16:33.707207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 11 16:16:33.707207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [parameters: {'id': 'ef070cdc8ed44830899cc8e37f68936e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 11 16:16:33.707207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ef070cdc8ed44830899cc8e37f68936e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ef070cdc8ed44830899cc8e37f68936e. Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 16:16:33.714990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 11 16:16:33.715810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application yield resource Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise self.value Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application session.commit() Aug 11 16:16:33.716819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.flush() Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application compat.raise_( Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 16:16:33.717747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 16:16:33.718656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 16:16:33.719474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [parameters: {'id': 'ef070cdc8ed44830899cc8e37f68936e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:33.722248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:33.724237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ef070cdc8ed44830899cc8e37f68936e. Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:33.726361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 481/947] 10.222.0.43 () {60 vars in 1140 bytes} [Thu Aug 11 16:16:33 2022] POST /identity/v3/projects => generated 262 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:33.733194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791abfe-3cc6-4adc-a508-3d033cca6c6f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:33.841314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791abfe-3cc6-4adc-a508-3d033cca6c6f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:33.878957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943e41a-4087-470a-974c-6c2559eb24e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:33.880038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e41a-4087-470a-974c-6c2559eb24e7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:33.880554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e41a-4087-470a-974c-6c2559eb24e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:33.881027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943e41a-4087-470a-974c-6c2559eb24e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:33.888084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943e41a-4087-470a-974c-6c2559eb24e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:33.889735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943e41a-4087-470a-974c-6c2559eb24e7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-2943e41a-4087-470a-974c-6c2559eb24e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:33.901818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:33.902752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:33.903642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:33.903642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:33.903642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:33.903642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:33.903642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:33.903642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:33.903642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:33.903642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:33.906047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 467/948] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:33.912566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:33.938236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:34.020949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18063a4-828a-4c3e-ac1d-4d24b15d6241 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:34.030518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffe384-a000-4724-924a-03943803aa00 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.030996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffe384-a000-4724-924a-03943803aa00 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.031073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffe384-a000-4724-924a-03943803aa00 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.032812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 468/949] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:34.038182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70179d56-8ef5-45ff-955c-3bf876067c75 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.040950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70179d56-8ef5-45ff-955c-3bf876067c75 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.040950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70179d56-8ef5-45ff-955c-3bf876067c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.065560 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-70179d56-8ef5-45ff-955c-3bf876067c75 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:34.066995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-70179d56-8ef5-45ff-955c-3bf876067c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:34.068046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:34.069046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.069402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.070887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.071639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:34.072019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:34.073751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:34.101251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 482/950] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:34.132059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:34.152539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 469/951] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:34.176583 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:34.277467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:34.300706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4b912-1126-4951-a4b5-db7009aa6acf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:34.337566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbff4e-b2fc-4bef-a4dc-6aa7d479e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:34.338192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbff4e-b2fc-4bef-a4dc-6aa7d479e50d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.338468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbff4e-b2fc-4bef-a4dc-6aa7d479e50d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.338733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbff4e-b2fc-4bef-a4dc-6aa7d479e50d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.349898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdbff4e-b2fc-4bef-a4dc-6aa7d479e50d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5bdbff4e-b2fc-4bef-a4dc-6aa7d479e50d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:34.354556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:34.355529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:34.356484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 483/952] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Aug 11 16:16:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:34.361750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70179d56-8ef5-45ff-955c-3bf876067c75 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:34.377692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26dd1-ae90-4e21-a76c-4c44b1dfcfab None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.378348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26dd1-ae90-4e21-a76c-4c44b1dfcfab None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.378841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26dd1-ae90-4e21-a76c-4c44b1dfcfab None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.380659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 484/953] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:34.385557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d3b8b-620f-433e-ae37-6431bd7f3e37 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.385837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d3b8b-620f-433e-ae37-6431bd7f3e37 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.386109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d3b8b-620f-433e-ae37-6431bd7f3e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.410980 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-026d3b8b-620f-433e-ae37-6431bd7f3e37 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:34.412821 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-026d3b8b-620f-433e-ae37-6431bd7f3e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:34.479880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70179d56-8ef5-45ff-955c-3bf876067c75 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:34.496056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 485/954] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:34.516595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:34.517442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.518510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.518697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d3b8b-620f-433e-ae37-6431bd7f3e37 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:34.519088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.519912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:34.520398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:34.521952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:34.530879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 470/955] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Aug 11 16:16:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:34.536364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:34.602489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:34.612525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:34.643375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d3b8b-620f-433e-ae37-6431bd7f3e37 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:34.671201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e8557-e9cc-4ec9-9a13-e92cc9cc9f26 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:34.686591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:34.687239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.687522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.687798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.688421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:34.691149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:34.702715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:34.703898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:34.704936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 486/956] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:34.709834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:34.720534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:34.721753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.722034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.722305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.733353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:34.734801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:34.749775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:34.750835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:34.751772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:34.751772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:34.751772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:34.751772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 471/957] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:34.761734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:34.831623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7ba35-8d46-426c-979f-b7d6e1eace99 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:34.878169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:34.878776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.879050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.879317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.879950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] RBAC: Adding query filter params (name=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:34.880279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:34.881885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:34.884063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:34.887492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:34.891295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fde41-6764-4c9d-b762-81decc3f8744 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:34.892474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 487/958] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:34.894074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:34.935139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:34.936153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:34.936658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:34.937145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:34.938001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:34.938508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:34.940632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:34.958522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 472/959] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:34.959735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.004122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc478d3-5a40-4c3c-9070-900b90e0b1af None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.040344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018f104-e89b-4a3c-a921-1c21352e18ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.041258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018f104-e89b-4a3c-a921-1c21352e18ee None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.041258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018f104-e89b-4a3c-a921-1c21352e18ee None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.041518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018f104-e89b-4a3c-a921-1c21352e18ee None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.042156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018f104-e89b-4a3c-a921-1c21352e18ee None admin] RBAC: Adding query filter params (type=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:35.042459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018f104-e89b-4a3c-a921-1c21352e18ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.043843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018f104-e89b-4a3c-a921-1c21352e18ee None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:35.055052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 488/960] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:35.073208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc7a30b-3155-41d5-bc3c-2bca805e54c7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.114839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.115861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.116390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.116885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.125659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.128254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:35.141678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:35.142800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 11 16:16:35.143840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:35.146663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66c004-0489-4d3a-a6f4-a5081f8f29d8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.146820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 473/961] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:16:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:35.146986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66c004-0489-4d3a-a6f4-a5081f8f29d8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.151671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66c004-0489-4d3a-a6f4-a5081f8f29d8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.151671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 489/962] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:35.153055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.154553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f4884-cfd2-45c9-86b7-3617a11085c6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.154941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f4884-cfd2-45c9-86b7-3617a11085c6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.155308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f4884-cfd2-45c9-86b7-3617a11085c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.176198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-374f4884-cfd2-45c9-86b7-3617a11085c6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:35.177573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-374f4884-cfd2-45c9-86b7-3617a11085c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:35.178818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:35.251342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:35.265893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 490/963] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:35.274095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3fddd-cb02-4194-a42e-31b5be7b9db3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.288651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f4884-cfd2-45c9-86b7-3617a11085c6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.322429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.323572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.323999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.324999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.326157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:35.326608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.328536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:35.353718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 474/964] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Aug 11 16:16:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:35.358823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.400839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f4884-cfd2-45c9-86b7-3617a11085c6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.449974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.451448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.451729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.452002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.452630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.453993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:35.460864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:35.466351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:35.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:35.474108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 491/965] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:35.476591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.490929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78264535-a510-4e3f-a63a-5f5ba9a0f646 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.526201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bec651-65f4-4eb5-bbef-f41b14e4546c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.526790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bec651-65f4-4eb5-bbef-f41b14e4546c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.527061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bec651-65f4-4eb5-bbef-f41b14e4546c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.527331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bec651-65f4-4eb5-bbef-f41b14e4546c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.542233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:35.590595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad062a-9bd6-4d6e-b1ce-41740d298009 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.614198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bec651-65f4-4eb5-bbef-f41b14e4546c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.615730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bec651-65f4-4eb5-bbef-f41b14e4546c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:35.630105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.631045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.631601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.632065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.633016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:35.633528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.635193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:35.645809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 492/966] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:35.647366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.701025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:35.729924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 475/967] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:35 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 16:16:35.731005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bec651-65f4-4eb5-bbef-f41b14e4546c None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.757981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962bbcf-6348-43b0-8986-54aba3a2d732 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.782694 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:35.796994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.797947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.798537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.799635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.800554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:35.801073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.802812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:35.816348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 493/968] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:35.818167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.842051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bec651-65f4-4eb5-bbef-f41b14e4546c None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.881148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.881693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.881974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.882246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.893476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.894993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:35.910759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:35.911773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:35.913348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:35.913348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:35.913348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:35.916629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 476/969] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:35.917526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:35.932338 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e681dc-06d1-45ca-b8f8-3124a69d9353 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:35.952077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:35.971585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9682e7f-54c2-49f5-adc9-679e66dff6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:35.972831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9682e7f-54c2-49f5-adc9-679e66dff6a3 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:35.973450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9682e7f-54c2-49f5-adc9-679e66dff6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:35.973916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9682e7f-54c2-49f5-adc9-679e66dff6a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:35.974777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9682e7f-54c2-49f5-adc9-679e66dff6a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:35.976604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9682e7f-54c2-49f5-adc9-679e66dff6a3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:36.008661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 494/970] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:16:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:16:36.037987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd6d83-84e1-45e8-af19-e19085796d31 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:36.088211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:36.089045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:36.089493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:36.089773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:36.090524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:36.090830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:36.092358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:36.107778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 477/971] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:36.114423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9682e7f-54c2-49f5-adc9-679e66dff6a3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:36.162760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:36.232558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9682e7f-54c2-49f5-adc9-679e66dff6a3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:36.271666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54902867-40d9-4b1a-bd2e-b88ac9fa11c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:36.272710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54902867-40d9-4b1a-bd2e-b88ac9fa11c0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:36.273189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54902867-40d9-4b1a-bd2e-b88ac9fa11c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:36.273651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54902867-40d9-4b1a-bd2e-b88ac9fa11c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:36.281120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54902867-40d9-4b1a-bd2e-b88ac9fa11c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:36.282814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54902867-40d9-4b1a-bd2e-b88ac9fa11c0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-54902867-40d9-4b1a-bd2e-b88ac9fa11c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:36.295010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:36.295938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:36.297283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:36.297283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:36.297283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:36.297283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:36.297283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:36.297283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:36.297283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:36.297283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:36.299955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 495/972] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:36.308159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:36.342427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d2dd-406d-45f0-9e1c-4f981b00a56d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:36.342768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d2dd-406d-45f0-9e1c-4f981b00a56d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:36.342977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46d2dd-406d-45f0-9e1c-4f981b00a56d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:36.344635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 496/973] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:36.350072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce724e2-80e6-4fa3-b333-c4a05dc19e83 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:36.350350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce724e2-80e6-4fa3-b333-c4a05dc19e83 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:36.350624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce724e2-80e6-4fa3-b333-c4a05dc19e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:36.374449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7ce724e2-80e6-4fa3-b333-c4a05dc19e83 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:36.376135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7ce724e2-80e6-4fa3-b333-c4a05dc19e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:36.403840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:36.421280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460184-431f-42fe-9ea4-887a66317559 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:36.464178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 497/974] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:36.467144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:36.468046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:36.468562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:36.469033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:36.469934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:36.470509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:36.472284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:36.484939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 478/975] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:36.487943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce724e2-80e6-4fa3-b333-c4a05dc19e83 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:36.491252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:36.526822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:36.607874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bf3ea-54bf-4df4-bef3-407545c30894 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:36.619255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce724e2-80e6-4fa3-b333-c4a05dc19e83 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:36.648334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:36.649029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:36.649376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:36.649716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:36.650521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] RBAC: Adding query filter params (role.id=34515f0849514506bafe075b65e3aa5e, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=d7291ccb171e44f89dabc99980312c3b) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:36.651014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:36.652723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:36.661164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 479/976] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:36 2022] GET /identity/v3/role_assignments?user.id=d7291ccb171e44f89dabc99980312c3b&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=34515f0849514506bafe075b65e3aa5e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:36.670133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:36.671045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:36.671452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:36.671848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:36.672755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:36.675223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:36.692762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.sql.core [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 16:16:36.692762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 11 16:16:36.692762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [parameters: {'user_id': '9957290ba4de42f7956b3067f0f57d6b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 11 16:16:36.692762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81428) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 11 16:16:36.700402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 16:16:36.701620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application yield resource Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application next(self.gen) Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.rollback() Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.force_reraise() Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise self.value Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application session.commit() Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 11 16:16:36.702865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._prepare_impl() Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.session.flush() Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._flush(objects) Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application compat.raise_( Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application flush_context.execute() Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rec.execute(self) Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 11 16:16:36.703978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 11 16:16:36.704947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result = self._query(query) Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application conn.query(q) Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application result.read() Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 11 16:16:36.706643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [parameters: {'user_id': '9957290ba4de42f7956b3067f0f57d6b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:36.707500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 11 16:16:36.708526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 11 16:16:36.710046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 11 16:16:36.710046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 11 16:16:36.710046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 11 16:16:36.710046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 11 16:16:36.710046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 11 16:16:36.710046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:36.717524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 498/977] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Aug 11 16:16:36 2022] POST /identity/v3/users => generated 160 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:36.722177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:36.738373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:36.838045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753dcba-085a-4131-a649-949740d739a7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:36.874944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d44766-1f1e-4bf1-a776-7a26290d7a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:36.875815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d44766-1f1e-4bf1-a776-7a26290d7a5d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:36.876261 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d44766-1f1e-4bf1-a776-7a26290d7a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:36.876670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d44766-1f1e-4bf1-a776-7a26290d7a5d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:36.885106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d44766-1f1e-4bf1-a776-7a26290d7a5d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:36.886731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d44766-1f1e-4bf1-a776-7a26290d7a5d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-23d44766-1f1e-4bf1-a776-7a26290d7a5d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:36.898302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:36.899491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 11 16:16:36.900695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:36.907417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 480/978] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:36.910089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:37.011356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:37.020350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d3103-52bd-4da6-9f89-7991ed695c86 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:37.067872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2551-9d87-4c5f-8ae1-f7b5dbd75bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:37.068847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2551-9d87-4c5f-8ae1-f7b5dbd75bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:37.069267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2551-9d87-4c5f-8ae1-f7b5dbd75bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:37.069695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2551-9d87-4c5f-8ae1-f7b5dbd75bd2 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:37.070533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f2551-9d87-4c5f-8ae1-f7b5dbd75bd2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:37.070972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f2551-9d87-4c5f-8ae1-f7b5dbd75bd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:37.072785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f2551-9d87-4c5f-8ae1-f7b5dbd75bd2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:37.116539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 499/979] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Aug 11 16:16:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:37.160206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fdc36-84ea-4920-ab4e-3dca6405d7da None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:37.160752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fdc36-84ea-4920-ab4e-3dca6405d7da None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:37.160752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fdc36-84ea-4920-ab4e-3dca6405d7da None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:37.162438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 481/980] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:37.168255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7179bb-554f-456f-b10f-460ba3d0feb4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:37.168541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7179bb-554f-456f-b10f-460ba3d0feb4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:37.168812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7179bb-554f-456f-b10f-460ba3d0feb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:37.189422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3c7179bb-554f-456f-b10f-460ba3d0feb4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:37.191668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3c7179bb-554f-456f-b10f-460ba3d0feb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:37.264547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 500/981] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:37.287196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fdc36-84ea-4920-ab4e-3dca6405d7da None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:37.362757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:37.397174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fdc36-84ea-4920-ab4e-3dca6405d7da None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:37.434916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37788570-e108-4a19-b972-7645995e28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:37.435525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788570-e108-4a19-b972-7645995e28bb None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:37.435800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788570-e108-4a19-b972-7645995e28bb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:37.436067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788570-e108-4a19-b972-7645995e28bb None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:37.436746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37788570-e108-4a19-b972-7645995e28bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:37.438112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37788570-e108-4a19-b972-7645995e28bb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:37.454132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 482/982] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:16:37 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:38.214399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d973d91-bb37-49f6-814a-64a1f8035a81 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:38.214911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d973d91-bb37-49f6-814a-64a1f8035a81 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:38.215022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d973d91-bb37-49f6-814a-64a1f8035a81 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:38.216482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 501/983] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:38.221528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa60d3-ce12-41f7-8281-57ee1162469b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:38.221911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa60d3-ce12-41f7-8281-57ee1162469b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:38.222082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa60d3-ce12-41f7-8281-57ee1162469b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:38.240345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-fdaa60d3-ce12-41f7-8281-57ee1162469b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:38.241540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fdaa60d3-ce12-41f7-8281-57ee1162469b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:38.317840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 483/984] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:38.340025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d973d91-bb37-49f6-814a-64a1f8035a81 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:38.424700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:38.445499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d973d91-bb37-49f6-814a-64a1f8035a81 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:38.481732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:38.482361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:38.482642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:38.482910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:38.483492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:38.484872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:38.494542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:38.495531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:38.496820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:38.503722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 502/985] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:38.503862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa60d3-ce12-41f7-8281-57ee1162469b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:38.560373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:38.607558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa60d3-ce12-41f7-8281-57ee1162469b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:38.643205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988d225-15cc-42bf-aac9-cf29ce176c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:38.644190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988d225-15cc-42bf-aac9-cf29ce176c03 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:38.644540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988d225-15cc-42bf-aac9-cf29ce176c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:38.644812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988d225-15cc-42bf-aac9-cf29ce176c03 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:38.645446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988d225-15cc-42bf-aac9-cf29ce176c03 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:38.645747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988d225-15cc-42bf-aac9-cf29ce176c03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:38.647103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988d225-15cc-42bf-aac9-cf29ce176c03 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:38.652060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 484/986] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:38.657645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:38.741968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0692be-7d61-46da-a304-b767a0eb18ea None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:38.742257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0692be-7d61-46da-a304-b767a0eb18ea None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:38.742523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0692be-7d61-46da-a304-b767a0eb18ea None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:38.744142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 485/987] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:38.748734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560dd9c-17c7-4a48-97ce-d8fc1552feff None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:38.749255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560dd9c-17c7-4a48-97ce-d8fc1552feff None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:38.749859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560dd9c-17c7-4a48-97ce-d8fc1552feff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:38.762193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263b6f3-b957-4311-9734-d5abd71b1ae6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:38.769839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3560dd9c-17c7-4a48-97ce-d8fc1552feff None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:38.771031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3560dd9c-17c7-4a48-97ce-d8fc1552feff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:38.796048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:38.796681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:38.796960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:38.797230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:38.797861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:38.798161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:38.800594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:38.805454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 503/988] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:38.811182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:38.823931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:38.845439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 486/989] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:38.867580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560dd9c-17c7-4a48-97ce-d8fc1552feff None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:38.920241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:38.922650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10629d60-63b0-40bd-945b-99b5da75acc4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:38.959316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:38.959937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:38.960262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:38.960537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:38.961371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] RBAC: Adding query filter params (interface=admin, service_id=8b8a1c4dd644495badf6877ad52521cc, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:38.961621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:38.963009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:38.972725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 504/990] 10.222.0.43 () {56 vars in 1249 bytes} [Thu Aug 11 16:16:38 2022] GET /identity/v3/endpoints?service_id=8b8a1c4dd644495badf6877ad52521cc&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:38.973739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:38.983462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560dd9c-17c7-4a48-97ce-d8fc1552feff None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:39.027354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:39.027988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:39.028299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:39.028570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] PATH_INFO: `/v3/services/image` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:39.029621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:39.030985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:39.041740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:39.042814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:39.043668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:39.050430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 487/991] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Aug 11 16:16:38 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:39.053940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:39.060497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:39.088714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fd72a-28af-4da8-b294-3d8e1281ef45 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:39.124463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b4ee-005d-4be2-ac16-11a3199bd87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:39.125091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b4ee-005d-4be2-ac16-11a3199bd87e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:39.125370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b4ee-005d-4be2-ac16-11a3199bd87e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:39.125644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b4ee-005d-4be2-ac16-11a3199bd87e None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:39.126247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539b4ee-005d-4be2-ac16-11a3199bd87e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:39.128173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539b4ee-005d-4be2-ac16-11a3199bd87e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:39.140190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 505/992] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:16:38 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:16:39.173533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9e736-6a35-46d7-979b-759265ee33cb None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:39.188642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:39.213601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d00c8-54d9-4c60-95ba-011d1ccd95b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:39.214217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d00c8-54d9-4c60-95ba-011d1ccd95b7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:39.214491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d00c8-54d9-4c60-95ba-011d1ccd95b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:39.214758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d00c8-54d9-4c60-95ba-011d1ccd95b7 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:39.215395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d00c8-54d9-4c60-95ba-011d1ccd95b7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:39.215692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d00c8-54d9-4c60-95ba-011d1ccd95b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:39.217135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d00c8-54d9-4c60-95ba-011d1ccd95b7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:39.222962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 488/993] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:39.229756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b4ee-005d-4be2-ac16-11a3199bd87e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:39.336649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b4ee-005d-4be2-ac16-11a3199bd87e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:39.371837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa3d6-ad52-4cb4-946d-529659a89130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:39.372490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa3d6-ad52-4cb4-946d-529659a89130 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:39.372770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa3d6-ad52-4cb4-946d-529659a89130 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:39.373043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa3d6-ad52-4cb4-946d-529659a89130 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:39.373675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafa3d6-ad52-4cb4-946d-529659a89130 None admin] RBAC: Adding query filter params (type=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:39.373976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafa3d6-ad52-4cb4-946d-529659a89130 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:39.375390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafa3d6-ad52-4cb4-946d-529659a89130 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:39.380229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 506/994] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:39.471391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dc8e4-d016-442d-ad2b-e8a088f1be04 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:39.471819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dc8e4-d016-442d-ad2b-e8a088f1be04 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:39.472110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dc8e4-d016-442d-ad2b-e8a088f1be04 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:39.473641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 489/995] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:39.479118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b736c-a415-4d08-b9ec-d5c20ca3c439 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:39.479406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b736c-a415-4d08-b9ec-d5c20ca3c439 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:39.479674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b736c-a415-4d08-b9ec-d5c20ca3c439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:39.486809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:39.500762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bf1b736c-a415-4d08-b9ec-d5c20ca3c439 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:39.502537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bf1b736c-a415-4d08-b9ec-d5c20ca3c439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:39.575650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 507/996] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:39.597631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dc8e4-d016-442d-ad2b-e8a088f1be04 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:39.707229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dc8e4-d016-442d-ad2b-e8a088f1be04 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:39.741657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:39.742294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:39.742572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:39.742840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:39.743425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:39.744801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:39.754109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:39.755098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:39.755992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 490/997] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:39.761309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b736c-a415-4d08-b9ec-d5c20ca3c439 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:39.866135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b736c-a415-4d08-b9ec-d5c20ca3c439 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:39.898940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:39.904894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:39.905566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:39.905850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:39.906120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:39.906748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:39.907049 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:39.909149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:39.918532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 508/998] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:39.919825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:39.966272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:40.025537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b92f835-6f5a-47fa-9dd7-fe931fe4c966 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:40.068252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:40.068864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:40.069145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:40.069416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:40.070046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:40.070348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:40.071704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:40.082293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 491/999] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:40.083498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:40.187126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe7213-e2c2-4480-97cd-6df872801a9d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:40.222025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473af02-605b-4378-98bd-52d479244916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:40.222607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473af02-605b-4378-98bd-52d479244916 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:40.222888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473af02-605b-4378-98bd-52d479244916 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:40.223156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473af02-605b-4378-98bd-52d479244916 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:40.223854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473af02-605b-4378-98bd-52d479244916 None admin] RBAC: Adding query filter params (interface=public, service_id=b62a18523a7f4e75a19a327f8252bc0d, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:40.224184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473af02-605b-4378-98bd-52d479244916 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:40.225529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473af02-605b-4378-98bd-52d479244916 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:40.234816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 509/1000] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Aug 11 16:16:40 2022] GET /identity/v3/endpoints?service_id=b62a18523a7f4e75a19a327f8252bc0d&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:40.236070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:40.350192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074ed4e-d7a1-4853-9876-ce40754a3d36 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:40.358310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:40.387491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b2b9a-b6df-4dec-8dd7-ac2201bd692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:40.388110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b2b9a-b6df-4dec-8dd7-ac2201bd692e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:40.388397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b2b9a-b6df-4dec-8dd7-ac2201bd692e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:40.388664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b2b9a-b6df-4dec-8dd7-ac2201bd692e None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:40.389260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b2b9a-b6df-4dec-8dd7-ac2201bd692e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:40.390608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b2b9a-b6df-4dec-8dd7-ac2201bd692e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:40.397948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 492/1001] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:16:40 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:16:41.183365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722df274-0971-422f-a44f-fb556d518531 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.183759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722df274-0971-422f-a44f-fb556d518531 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.183941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722df274-0971-422f-a44f-fb556d518531 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.185438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 510/1002] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:41.191371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa861a-cbbd-4870-8c98-bce723160c62 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.191654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa861a-cbbd-4870-8c98-bce723160c62 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.191924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa861a-cbbd-4870-8c98-bce723160c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.209724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-69fa861a-cbbd-4870-8c98-bce723160c62 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:41.210894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-69fa861a-cbbd-4870-8c98-bce723160c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:41.278550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 493/1003] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:41.301891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722df274-0971-422f-a44f-fb556d518531 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:41.365228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:41.408548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f384d-7c56-48c8-ba10-a54bdbb8bcc0 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.409457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f384d-7c56-48c8-ba10-a54bdbb8bcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.409743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f384d-7c56-48c8-ba10-a54bdbb8bcc0 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.411210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 494/1004] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:41.411344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722df274-0971-422f-a44f-fb556d518531 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:41.417399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3016ff0-1d91-40f4-9864-8219f4af53b6 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.417918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3016ff0-1d91-40f4-9864-8219f4af53b6 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.418398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3016ff0-1d91-40f4-9864-8219f4af53b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.438587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-b3016ff0-1d91-40f4-9864-8219f4af53b6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:41.440674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b3016ff0-1d91-40f4-9864-8219f4af53b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:41.447709 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:41.448368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.448646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.448911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.449489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:41.450837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:41.459907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:41.460988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:41.465693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 511/1005] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Aug 11 16:16:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:41.467095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:41.519087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 495/1006] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:41.542615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3016ff0-1d91-40f4-9864-8219f4af53b6 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:41.577685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:41.589443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6792a3-6ef8-4577-bcbd-497a5df475d3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:41.614838 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:41.630058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:41.630608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.630894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.631166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.631805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:41.632200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:41.633565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:41.638431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 512/1007] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:41.645106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:41.655890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3016ff0-1d91-40f4-9864-8219f4af53b6 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:41.695368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e313f-33af-4ab5-b65b-b0a571abeb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:41.696061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e313f-33af-4ab5-b65b-b0a571abeb9d None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.696378 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e313f-33af-4ab5-b65b-b0a571abeb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.696652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e313f-33af-4ab5-b65b-b0a571abeb9d None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.697290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e313f-33af-4ab5-b65b-b0a571abeb9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:41.698829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e313f-33af-4ab5-b65b-b0a571abeb9d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:41.716307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 496/1008] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:16:41 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:16:41.749236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc378b3-47d6-4e59-a836-66bca3eaccc3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:41.795525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7be25-beb3-4d39-b394-e6a8755e167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:41.796165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7be25-beb3-4d39-b394-e6a8755e167b None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.796447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7be25-beb3-4d39-b394-e6a8755e167b None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.796719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7be25-beb3-4d39-b394-e6a8755e167b None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.797359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7be25-beb3-4d39-b394-e6a8755e167b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:41.797664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7be25-beb3-4d39-b394-e6a8755e167b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:41.799151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7be25-beb3-4d39-b394-e6a8755e167b None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:41.810234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 513/1009] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Aug 11 16:16:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:41.811753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e313f-33af-4ab5-b65b-b0a571abeb9d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:41.924088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:41.935615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e313f-33af-4ab5-b65b-b0a571abeb9d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:41.980294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ac37b-80eb-4a04-8894-f6d6dd46cd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:41.980917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ac37b-80eb-4a04-8894-f6d6dd46cd7f None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:41.981201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ac37b-80eb-4a04-8894-f6d6dd46cd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:41.981479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ac37b-80eb-4a04-8894-f6d6dd46cd7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:41.982149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ac37b-80eb-4a04-8894-f6d6dd46cd7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:41.983559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ac37b-80eb-4a04-8894-f6d6dd46cd7f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:42.005655 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 497/1010] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:16:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:42.453207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c148d-250c-4092-8d1d-a7b78a69981a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:42.453755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c148d-250c-4092-8d1d-a7b78a69981a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:42.453755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c148d-250c-4092-8d1d-a7b78a69981a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:42.455841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 514/1011] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:42.461415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d078a-ba58-4fb5-b2e9-62390a8decdb None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:42.461865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d078a-ba58-4fb5-b2e9-62390a8decdb None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:42.462082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d078a-ba58-4fb5-b2e9-62390a8decdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:42.481649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e93d078a-ba58-4fb5-b2e9-62390a8decdb None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:42.482852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e93d078a-ba58-4fb5-b2e9-62390a8decdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:42.557913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 498/1012] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:42 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:42.580981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c148d-250c-4092-8d1d-a7b78a69981a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:42.631986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:42.685170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c148d-250c-4092-8d1d-a7b78a69981a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:42.719488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:42.720036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:42.720338 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:42.721167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:42.721755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:42.723103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:42.731795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:42.732967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 11 16:16:42.733826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:42.735213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 515/1013] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:42.738967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d078a-ba58-4fb5-b2e9-62390a8decdb None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:42.816237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:42.841301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d078a-ba58-4fb5-b2e9-62390a8decdb None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:42.875793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:42.876428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:42.877225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:42.877676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:42.878316 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:42.878617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:42.879985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:42.889491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 499/1014] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:42.890529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:42.993698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5b4e-a29b-4721-9d60-e6a09cd33f38 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:43.006782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:43.029274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420808b4-a7e5-4732-894c-a6b6e8fdb8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:43.029885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420808b4-a7e5-4732-894c-a6b6e8fdb8b4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:43.030164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420808b4-a7e5-4732-894c-a6b6e8fdb8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:43.030434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420808b4-a7e5-4732-894c-a6b6e8fdb8b4 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:43.031059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420808b4-a7e5-4732-894c-a6b6e8fdb8b4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:43.031363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420808b4-a7e5-4732-894c-a6b6e8fdb8b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:43.032749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420808b4-a7e5-4732-894c-a6b6e8fdb8b4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:43.042635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 516/1015] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Aug 11 16:16:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:43.043928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:43.149268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bcc63-fa18-4182-ba2f-47d73ac92d1e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:43.184225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda0ab2-a953-4ebd-8909-25850f08ceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:43.185088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda0ab2-a953-4ebd-8909-25850f08ceb1 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:43.185557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda0ab2-a953-4ebd-8909-25850f08ceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:43.185997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda0ab2-a953-4ebd-8909-25850f08ceb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:43.186848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda0ab2-a953-4ebd-8909-25850f08ceb1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:43.188404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda0ab2-a953-4ebd-8909-25850f08ceb1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:43.198516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:43.212483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 500/1016] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:16:43 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:43.836684 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fc93b-aa5b-43be-b4ae-17a4b69d8eef None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:43.837269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fc93b-aa5b-43be-b4ae-17a4b69d8eef None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:43.837269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fc93b-aa5b-43be-b4ae-17a4b69d8eef None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:43.838764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 517/1017] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:43.844065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fdad2-4357-4795-a783-f47046af794d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:43.844490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fdad2-4357-4795-a783-f47046af794d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:43.844656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fdad2-4357-4795-a783-f47046af794d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:43.863523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-cf9fdad2-4357-4795-a783-f47046af794d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:43.864722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-cf9fdad2-4357-4795-a783-f47046af794d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:43.940806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 501/1018] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:43.963900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fc93b-aa5b-43be-b4ae-17a4b69d8eef None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:44.061701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53879-a78b-4629-b96f-760a84d01e27 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.061990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53879-a78b-4629-b96f-760a84d01e27 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.062266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53879-a78b-4629-b96f-760a84d01e27 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.063822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 502/1019] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:44.069639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbe163-a47c-45d1-bb1d-4847f847a85f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.069924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbe163-a47c-45d1-bb1d-4847f847a85f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.070199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbe163-a47c-45d1-bb1d-4847f847a85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.088782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-fdbbe163-a47c-45d1-bb1d-4847f847a85f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:44.089904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fdbbe163-a47c-45d1-bb1d-4847f847a85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:44.105346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fc93b-aa5b-43be-b4ae-17a4b69d8eef None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:44.147274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:44.148274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.148545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.148813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] PATH_INFO: `/v3/services/image` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.149388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:44.150741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:44.159829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:44.160885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:44.162496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 518/1020] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Aug 11 16:16:43 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:44.164974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 503/1021] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:44.169195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:44.187400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbe163-a47c-45d1-bb1d-4847f847a85f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:44.216575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:44.245273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:44.278992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd26bf-823f-46f4-9225-a4c3ec198e81 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:44.296583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbe163-a47c-45d1-bb1d-4847f847a85f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:44.316751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:44.317363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.317645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.317910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.318538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:44.318839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:44.320212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:44.330856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 519/1022] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:44.331909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:44.333416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cae2b1-a09f-4661-96bc-e9568e9c8728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:44.334129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cae2b1-a09f-4661-96bc-e9568e9c8728 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.334677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cae2b1-a09f-4661-96bc-e9568e9c8728 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.334947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cae2b1-a09f-4661-96bc-e9568e9c8728 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.336966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cae2b1-a09f-4661-96bc-e9568e9c8728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:44.338380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cae2b1-a09f-4661-96bc-e9568e9c8728 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:44.344733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 504/1023] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:44.346373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cae2b1-a09f-4661-96bc-e9568e9c8728 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:44.438557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee93b5b-acd4-46f2-9dbd-7e55282355c2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:44.442351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:44.457586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cae2b1-a09f-4661-96bc-e9568e9c8728 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:44.481300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:44.486319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:44.487801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.488077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.488549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.489207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:44.489819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:44.491182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:44.496480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 520/1024] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:44.501365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b9620-d376-44df-87b1-03db66ad4e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:44.502855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:44.503217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b9620-d376-44df-87b1-03db66ad4e12 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.503614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b9620-d376-44df-87b1-03db66ad4e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.503896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b9620-d376-44df-87b1-03db66ad4e12 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.504832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b9620-d376-44df-87b1-03db66ad4e12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:44.506152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b9620-d376-44df-87b1-03db66ad4e12 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:44.535678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 505/1025] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:16:44 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:16:44.638650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb7590-a131-4df2-b67d-f463e57279e4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:44.676674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52597f00-c6c8-48e0-be15-cd33a0437516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:44.677289 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52597f00-c6c8-48e0-be15-cd33a0437516 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.677564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52597f00-c6c8-48e0-be15-cd33a0437516 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.677831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52597f00-c6c8-48e0-be15-cd33a0437516 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.678540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52597f00-c6c8-48e0-be15-cd33a0437516 None admin] RBAC: Adding query filter params (interface=public, service_id=2572b315378e47b292f836a03751ba51, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:44.678841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52597f00-c6c8-48e0-be15-cd33a0437516 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:44.680229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52597f00-c6c8-48e0-be15-cd33a0437516 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:44.686062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 521/1026] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Aug 11 16:16:44 2022] GET /identity/v3/endpoints?service_id=2572b315378e47b292f836a03751ba51&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:44.692532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b9620-d376-44df-87b1-03db66ad4e12 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:44.805914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b9620-d376-44df-87b1-03db66ad4e12 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:44.841531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02e41d-94f0-4b3d-a403-e9983a39f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:44.842141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02e41d-94f0-4b3d-a403-e9983a39f0ae None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:44.842415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02e41d-94f0-4b3d-a403-e9983a39f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:44.842682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02e41d-94f0-4b3d-a403-e9983a39f0ae None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:44.843289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02e41d-94f0-4b3d-a403-e9983a39f0ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:44.844676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02e41d-94f0-4b3d-a403-e9983a39f0ae None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:44.851967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 506/1027] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:16:44 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:16:45.380670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27354-c5af-4708-aa58-af1608d5de74 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:45.381186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27354-c5af-4708-aa58-af1608d5de74 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:45.381186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b27354-c5af-4708-aa58-af1608d5de74 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:45.382928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 522/1028] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:45.387755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41830c84-17a5-4099-b4ff-ca7beed29149 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:45.388047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41830c84-17a5-4099-b4ff-ca7beed29149 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:45.388359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41830c84-17a5-4099-b4ff-ca7beed29149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:45.407434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-41830c84-17a5-4099-b4ff-ca7beed29149 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:45.409232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-41830c84-17a5-4099-b4ff-ca7beed29149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:45.437139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:45.477320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 507/1029] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:45.500600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27354-c5af-4708-aa58-af1608d5de74 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:45.535336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:45.618507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b27354-c5af-4708-aa58-af1608d5de74 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:45.653179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347f281-8d7b-4428-aefd-ccc80bceab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:45.653723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f281-8d7b-4428-aefd-ccc80bceab90 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:45.653997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f281-8d7b-4428-aefd-ccc80bceab90 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:45.654261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f281-8d7b-4428-aefd-ccc80bceab90 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:45.667089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347f281-8d7b-4428-aefd-ccc80bceab90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c347f281-8d7b-4428-aefd-ccc80bceab90 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:45.672363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:45.673337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:45.674312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 523/1030] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Aug 11 16:16:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:45.681368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41830c84-17a5-4099-b4ff-ca7beed29149 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:45.782067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41830c84-17a5-4099-b4ff-ca7beed29149 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:45.815664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:45.816291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:45.816568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:45.816831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:45.817483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:45.817778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:45.819119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:45.832125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 508/1031] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Aug 11 16:16:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:45.834210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347f281-8d7b-4428-aefd-ccc80bceab90 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:45.932274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:45.943824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347f281-8d7b-4428-aefd-ccc80bceab90 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:45.979536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:45.980214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:45.980501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:45.980772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:45.993443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:45.994904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:46.010481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:46.011442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:46.012902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:46.012902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:46.012902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:46.016391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 524/1032] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:46.018282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:46.051116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:46.130257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c7bb8-ee92-4702-ad02-4fe3057297cd None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:46.164487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:46.165084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.165358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.165625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:46.166258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:46.166554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:46.167890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:46.180334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 509/1033] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:46.185923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:46.288598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd5d4-a3b5-4c62-b1e4-a51d08407dbe None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:46.323881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:46.324539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.324820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.325082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:46.332657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:46.334088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:46.344904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:46.345972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:46.351636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 525/1034] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:16:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:46.353962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:46.445111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:46.458502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623c93b-f4cb-45d5-9205-5345667d6517 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:46.499773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57478db7-8eff-4867-bf3f-183eb447ffe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:46.500582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57478db7-8eff-4867-bf3f-183eb447ffe3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.500954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57478db7-8eff-4867-bf3f-183eb447ffe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.501319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57478db7-8eff-4867-bf3f-183eb447ffe3 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:46.502714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57478db7-8eff-4867-bf3f-183eb447ffe3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:46.503111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57478db7-8eff-4867-bf3f-183eb447ffe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:46.504646 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57478db7-8eff-4867-bf3f-183eb447ffe3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:46.526675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 510/1035] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:16:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:46.527942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:46.531964 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:46.625296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac826fb-ee70-4733-a2ef-2517abc39d65 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.625581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac826fb-ee70-4733-a2ef-2517abc39d65 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.625857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac826fb-ee70-4733-a2ef-2517abc39d65 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:46.628273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 511/1036] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:46.630542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20341e91-e9f6-49e9-9e04-d9d7e9825e5f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:46.632464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c35788-4167-4043-a2b8-ea5b14084e23 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.632748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c35788-4167-4043-a2b8-ea5b14084e23 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.633025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c35788-4167-4043-a2b8-ea5b14084e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:46.651389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c9c35788-4167-4043-a2b8-ea5b14084e23 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:46.652614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c9c35788-4167-4043-a2b8-ea5b14084e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:46.667429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417353b1-25b2-4961-b07e-5825306b5ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:46.668188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417353b1-25b2-4961-b07e-5825306b5ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.668560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417353b1-25b2-4961-b07e-5825306b5ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.668890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417353b1-25b2-4961-b07e-5825306b5ce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:46.723389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 512/1037] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:46.742982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417353b1-25b2-4961-b07e-5825306b5ce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:46.744593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417353b1-25b2-4961-b07e-5825306b5ce5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:46.747472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:46.747940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c35788-4167-4043-a2b8-ea5b14084e23 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:46.828612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:46.852293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 526/1038] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 16:16:46.855301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c35788-4167-4043-a2b8-ea5b14084e23 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:46.858813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417353b1-25b2-4961-b07e-5825306b5ce5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:46.891577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e125c1-ab36-4119-bae5-4085f084beb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:46.892236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e125c1-ab36-4119-bae5-4085f084beb0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.893053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e125c1-ab36-4119-bae5-4085f084beb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.893319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e125c1-ab36-4119-bae5-4085f084beb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:46.894543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e125c1-ab36-4119-bae5-4085f084beb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:46.895977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e125c1-ab36-4119-bae5-4085f084beb0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:46.897879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 513/1039] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:46.899052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53358ba9-1c9a-469b-bfa5-e9138c7fa5bd None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.899336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53358ba9-1c9a-469b-bfa5-e9138c7fa5bd None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.899608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53358ba9-1c9a-469b-bfa5-e9138c7fa5bd None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:46.900977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 514/1040] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:46.905117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53358ba9-1c9a-469b-bfa5-e9138c7fa5bd None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:46.960091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417353b1-25b2-4961-b07e-5825306b5ce5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:46.962371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:46.993714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63248ca-d986-4703-9e7f-b5a685f8412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:46.994264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63248ca-d986-4703-9e7f-b5a685f8412e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:46.994541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63248ca-d986-4703-9e7f-b5a685f8412e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:46.994810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63248ca-d986-4703-9e7f-b5a685f8412e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.004625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63248ca-d986-4703-9e7f-b5a685f8412e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.005195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53358ba9-1c9a-469b-bfa5-e9138c7fa5bd None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.006050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63248ca-d986-4703-9e7f-b5a685f8412e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e63248ca-d986-4703-9e7f-b5a685f8412e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:47.020904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:47.021943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:47.022896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:47.022896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:47.022896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:47.022896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 527/1041] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:47.023285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a06a80-80f8-4684-9892-de7226cc02d2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.023566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a06a80-80f8-4684-9892-de7226cc02d2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.023832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a06a80-80f8-4684-9892-de7226cc02d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.041286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-01a06a80-80f8-4684-9892-de7226cc02d2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:47.042446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-01a06a80-80f8-4684-9892-de7226cc02d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:47.044000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee894e7-22e4-4fc9-84db-f61357e3870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.044729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee894e7-22e4-4fc9-84db-f61357e3870b None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.045010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee894e7-22e4-4fc9-84db-f61357e3870b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.045282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee894e7-22e4-4fc9-84db-f61357e3870b None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.045932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee894e7-22e4-4fc9-84db-f61357e3870b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.047351 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee894e7-22e4-4fc9-84db-f61357e3870b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.065756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:47.076513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 515/1042] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:16:46 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:47.077717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee894e7-22e4-4fc9-84db-f61357e3870b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:47.121251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 528/1043] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:47.145909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a06a80-80f8-4684-9892-de7226cc02d2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:47.196649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee894e7-22e4-4fc9-84db-f61357e3870b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.237723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.238431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.238699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.238888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.239524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:47.239823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.241216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.245268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:47.255538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 516/1044] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:47.261420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:47.264588 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a06a80-80f8-4684-9892-de7226cc02d2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.306353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.307000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.307281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.307556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] PATH_INFO: `/v3/services/image` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.308172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.312215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:47.321750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:47.322997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 11 16:16:47.324065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:47.324992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 529/1045] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Aug 11 16:16:47 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:47.330336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:47.337480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:47.387740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cbbde1-6163-498a-b7d0-29927fb2cade None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.426867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.427501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.427781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.428050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.435036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.435539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c540a9b-f793-4eb6-be88-cd6ad3c55777 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.436530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:47.448841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:47.449803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:47.450720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 517/1046] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:47.455604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:47.474793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.475403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.475686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.475959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.476624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] RBAC: Adding query filter params (name=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:47.476928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.478285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.484142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 530/1047] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:47.489708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:47.508723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:47.563771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db5de3-da60-41f6-b9d5-686ffea841cc None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.583857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:47.602646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6260864-c708-4e73-a9da-ad977e984d54 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.608437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.609123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.609471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.609812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.610585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:47.610957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.613140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.630574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 518/1048] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:47.631703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:47.644135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.644755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.645033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.645306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.645939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] RBAC: Adding query filter params (type=image) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:47.646240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.647603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.656961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 531/1049] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:47.657983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:47.738128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:47.743111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c5df5-7574-4c6a-890d-5004bd6a0457 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.761301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598479d-060d-4e63-8f15-98bf82b93c64 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:47.778866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45faf7-9f73-45ed-bde6-0e6c2e2d99a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.779600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45faf7-9f73-45ed-bde6-0e6c2e2d99a7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.779984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45faf7-9f73-45ed-bde6-0e6c2e2d99a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.780382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45faf7-9f73-45ed-bde6-0e6c2e2d99a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.781212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45faf7-9f73-45ed-bde6-0e6c2e2d99a7 None admin] RBAC: Adding query filter params (role.id=34515f0849514506bafe075b65e3aa5e, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=2e3819e78802469c83c09a1cdd865f58) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:47.781604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45faf7-9f73-45ed-bde6-0e6c2e2d99a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.783201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45faf7-9f73-45ed-bde6-0e6c2e2d99a7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.800129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 519/1050] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:47 2022] GET /identity/v3/role_assignments?user.id=2e3819e78802469c83c09a1cdd865f58&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=34515f0849514506bafe075b65e3aa5e => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:47.807616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc10b94-c797-4438-8e69-71f2d1f63434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:47.808452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10b94-c797-4438-8e69-71f2d1f63434 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:47.808837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10b94-c797-4438-8e69-71f2d1f63434 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:47.809218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc10b94-c797-4438-8e69-71f2d1f63434 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:47.810007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc10b94-c797-4438-8e69-71f2d1f63434 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:47.811538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc10b94-c797-4438-8e69-71f2d1f63434 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:47.837334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 532/1051] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:16:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:16:49.154670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3dfe8-83f2-4b09-8aed-d5a3095766d9 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.155218 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3dfe8-83f2-4b09-8aed-d5a3095766d9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.155218 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3dfe8-83f2-4b09-8aed-d5a3095766d9 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.156700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 520/1052] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:49.162220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc806135-912d-44e9-b0b4-1e4faadafe26 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.162602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc806135-912d-44e9-b0b4-1e4faadafe26 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.162780 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc806135-912d-44e9-b0b4-1e4faadafe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.180580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-bc806135-912d-44e9-b0b4-1e4faadafe26 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:49.181748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bc806135-912d-44e9-b0b4-1e4faadafe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:49.260534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 533/1053] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:49.284657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3dfe8-83f2-4b09-8aed-d5a3095766d9 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:49.331242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:49.386121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3dfe8-83f2-4b09-8aed-d5a3095766d9 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:49.419322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e020fe-6722-4c08-b25f-47dd39e9394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:49.419944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e020fe-6722-4c08-b25f-47dd39e9394d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.420246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e020fe-6722-4c08-b25f-47dd39e9394d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.420511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e020fe-6722-4c08-b25f-47dd39e9394d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.431149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e020fe-6722-4c08-b25f-47dd39e9394d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-f1e020fe-6722-4c08-b25f-47dd39e9394d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:49.435757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:49.436990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:49.437844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:49.440828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 521/1054] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:49.446804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc806135-912d-44e9-b0b4-1e4faadafe26 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:49.507487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:49.551582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc806135-912d-44e9-b0b4-1e4faadafe26 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:49.586098 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:49.586621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.586894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.587160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.587813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:49.588119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:49.589475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:49.608137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 534/1055] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:49.610873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e020fe-6722-4c08-b25f-47dd39e9394d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:49.714139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e020fe-6722-4c08-b25f-47dd39e9394d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:49.716503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:49.752739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b03258-7f8d-4fdf-86f5-0c306673a72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:49.753370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b03258-7f8d-4fdf-86f5-0c306673a72e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.753641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b03258-7f8d-4fdf-86f5-0c306673a72e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.753914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b03258-7f8d-4fdf-86f5-0c306673a72e None admin] PATH_INFO: `/v3/users/608f1e7e96024330a184d7846fcdc902` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.767384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b03258-7f8d-4fdf-86f5-0c306673a72e None admin] RBAC: Authorizing `identity:get_user(user_id=608f1e7e96024330a184d7846fcdc902)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:49.768866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b03258-7f8d-4fdf-86f5-0c306673a72e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:49.775163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 522/1056] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:16:49 2022] GET /identity/v3/users/608f1e7e96024330a184d7846fcdc902 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:49.776474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:49.860821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925efdb-428f-49cf-89a3-976d02e07bd7 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.861071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925efdb-428f-49cf-89a3-976d02e07bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.862419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925efdb-428f-49cf-89a3-976d02e07bd7 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.863905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 523/1057] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:49.868767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b401f40-5e9f-4b45-9d26-3715c46073aa None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.869052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b401f40-5e9f-4b45-9d26-3715c46073aa None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.869318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b401f40-5e9f-4b45-9d26-3715c46073aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.885174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24eb99-86a2-46e4-9153-aad852c98ee6 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:49.887579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-9b401f40-5e9f-4b45-9d26-3715c46073aa None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:49.888800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9b401f40-5e9f-4b45-9d26-3715c46073aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:49.909629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:49.923531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8500-babc-439e-a70b-310c184c7900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:49.924254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8500-babc-439e-a70b-310c184c7900 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.924521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8500-babc-439e-a70b-310c184c7900 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.924792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8500-babc-439e-a70b-310c184c7900 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.958877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 524/1058] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:49.960146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05629d-c27e-41ea-96fb-c1e266d7b9c9 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.960430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05629d-c27e-41ea-96fb-c1e266d7b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.960707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05629d-c27e-41ea-96fb-c1e266d7b9c9 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.964129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 525/1059] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:49.967059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2f78-8e86-4cdf-9a72-cbe813514e10 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:49.967528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2f78-8e86-4cdf-9a72-cbe813514e10 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:49.968120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2f78-8e86-4cdf-9a72-cbe813514e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:49.987662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-121a2f78-8e86-4cdf-9a72-cbe813514e10 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:49.988888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-121a2f78-8e86-4cdf-9a72-cbe813514e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:49.995369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8500-babc-439e-a70b-310c184c7900 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:49.996847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8500-babc-439e-a70b-310c184c7900 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.056498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 526/1060] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:50.057060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a2f78-8e86-4cdf-9a72-cbe813514e10 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.084918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:50.104187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 535/1061] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:16:50.104187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8500-babc-439e-a70b-310c184c7900 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.125533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:50.163928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a2f78-8e86-4cdf-9a72-cbe813514e10 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.196818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ddb9ee-f807-48d6-bb3a-13e1f7f4b08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.197443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddb9ee-f807-48d6-bb3a-13e1f7f4b08b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.197721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddb9ee-f807-48d6-bb3a-13e1f7f4b08b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.197991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddb9ee-f807-48d6-bb3a-13e1f7f4b08b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.200269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8500-babc-439e-a70b-310c184c7900 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.208641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ddb9ee-f807-48d6-bb3a-13e1f7f4b08b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-c1ddb9ee-f807-48d6-bb3a-13e1f7f4b08b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:50.213271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:50.214241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 527/1062] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:50.215104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ddb9ee-f807-48d6-bb3a-13e1f7f4b08b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.238308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93231b89-6f66-4768-9626-7f600e5dc1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.238909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93231b89-6f66-4768-9626-7f600e5dc1ef None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.239182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93231b89-6f66-4768-9626-7f600e5dc1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.239448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93231b89-6f66-4768-9626-7f600e5dc1ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.295696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:50.305692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93231b89-6f66-4768-9626-7f600e5dc1ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:50.307169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93231b89-6f66-4768-9626-7f600e5dc1ef None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.313860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ddb9ee-f807-48d6-bb3a-13e1f7f4b08b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.333430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:50.348398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc5f2f-c31c-4086-97c5-ca5d665cd275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.349025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc5f2f-c31c-4086-97c5-ca5d665cd275 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.349304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc5f2f-c31c-4086-97c5-ca5d665cd275 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.349597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc5f2f-c31c-4086-97c5-ca5d665cd275 None admin] PATH_INFO: `/v3/projects/6d112dc320a14edebd79c3863e6d2f01` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.350728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc5f2f-c31c-4086-97c5-ca5d665cd275 None admin] RBAC: Authorizing `identity:get_project(project_id=6d112dc320a14edebd79c3863e6d2f01)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:50.352176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc5f2f-c31c-4086-97c5-ca5d665cd275 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.353804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 528/1063] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/projects/6d112dc320a14edebd79c3863e6d2f01 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:50.354552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc5f2f-c31c-4086-97c5-ca5d665cd275 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.413726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 536/1064] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:16:50.414768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93231b89-6f66-4768-9626-7f600e5dc1ef None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.453371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc5f2f-c31c-4086-97c5-ca5d665cd275 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.487200 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.487796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.488073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.488377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.489040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:50.489341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:50.490692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.499379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 529/1065] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:50.500606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.516141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:50.521021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93231b89-6f66-4768-9626-7f600e5dc1ef None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.546621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:50.558643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.559252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.559528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.559803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.560569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=6d112dc320a14edebd79c3863e6d2f01, user.id=608f1e7e96024330a184d7846fcdc902) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:50.560875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:50.562301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.568373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 537/1066] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/role_assignments?user.id=608f1e7e96024330a184d7846fcdc902&scope.project.id=6d112dc320a14edebd79c3863e6d2f01&role.id=29fd031edd2b4f65aed8942853150f92 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:50.569145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.605903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f449d7f-e03a-4dde-b54a-ba4cc2f3af58 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.648628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.649275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.649549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.649820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.662040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:50.663496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:50.678678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:50.679892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:50.680868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:50.680868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:50.680868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:50.680868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 530/1067] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:50.685865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.697397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb54c0-1cb5-4765-afa5-f5c778f0be60 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.736261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.736900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.737181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.737448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.748185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:50.749624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.759870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:50.764675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:50.765678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:50.766587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:50.766587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:50.766587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:50.766587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 538/1068] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:50.771896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:50.805712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9d79f-a6a7-4651-a86b-4ad2f295418e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.815045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:50.841367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.841963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.842238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.842511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.843151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:50.843452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:50.844830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.857935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 531/1069] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:50.887109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2de3b-7231-4388-9814-2a87f5c6f663 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:50.927129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:50.927755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:50.928037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:50.928341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:50.929066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:50.929316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:50.930712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:50.943900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 539/1070] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:50.949389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:51.068465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9b3fb-5b99-49f9-a70d-c2869bf78d93 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:51.094799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:51.106684 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:51.109957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:51.109957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:51.109957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:51.115805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:51.117286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:51.132350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:51.133365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:51.134401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 532/1071] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:16:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:51.139411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:51.242636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:51.247027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7b6be-54fd-4d39-8a80-09becd797217 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:51.281018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:51.281622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:51.281898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:51.282163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:51.282931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:51.283242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:51.284676 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:51.301375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 540/1072] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:16:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:51.306999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:51.405906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3d5fc-83fc-4ec4-afbc-5635805027f0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:51.440514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:51.441277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:51.441559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:51.441841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:51.451958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:51.453421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:51.467488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:51.473586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:51.474570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:51.474570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:51.474570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:51.474570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 533/1073] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:51.474817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:51.572637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77489ce7-e07c-4b4d-84c0-51780c91b7f1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:51.605590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:51.606196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:51.606474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:51.606747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:51.607399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:51.607702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:51.609095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:51.625666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 541/1074] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:51.627595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:51.663003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:51.728967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c22ca2-f5ef-4e69-b1aa-982c7466ed73 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:51.762985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:51.763600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:51.763880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:51.764189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:51.770793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:51.772264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:51.783682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:51.784781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:51.785981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 534/1075] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:51.790577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:51.809216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:51.890064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e92b8-27a0-4188-8d78-7353ef12ca79 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:51.922164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969082d0-b1a8-4db4-a994-fffaacb761df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:51.922763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969082d0-b1a8-4db4-a994-fffaacb761df None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:51.923043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969082d0-b1a8-4db4-a994-fffaacb761df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:51.923316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969082d0-b1a8-4db4-a994-fffaacb761df None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:51.924658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969082d0-b1a8-4db4-a994-fffaacb761df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:51.924960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969082d0-b1a8-4db4-a994-fffaacb761df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:51.926370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969082d0-b1a8-4db4-a994-fffaacb761df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:51.938752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 542/1076] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:51.945062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:52.038904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:52.045941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b399d3-62f2-4216-9560-1f45c0fdb5af None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:52.080720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb3287-6d4d-42af-bdef-b52e458a1881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:52.081426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb3287-6d4d-42af-bdef-b52e458a1881 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:52.081712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb3287-6d4d-42af-bdef-b52e458a1881 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:52.081966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb3287-6d4d-42af-bdef-b52e458a1881 None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/2e3819e78802469c83c09a1cdd865f58/roles/34515f0849514506bafe075b65e3aa5e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:52.084061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb3287-6d4d-42af-bdef-b52e458a1881 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=2e3819e78802469c83c09a1cdd865f58, role_id=34515f0849514506bafe075b65e3aa5e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:52.085503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb3287-6d4d-42af-bdef-b52e458a1881 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:52.097611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 535/1077] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:16:51 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/2e3819e78802469c83c09a1cdd865f58/roles/34515f0849514506bafe075b65e3aa5e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:16:52.591463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f49967-93a6-4d84-b335-0029d1cd75b6 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:52.591991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f49967-93a6-4d84-b335-0029d1cd75b6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:52.591991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f49967-93a6-4d84-b335-0029d1cd75b6 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:52.593366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 543/1078] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:52.599352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44ff1-426d-415f-9747-57225d3247be None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:52.599760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44ff1-426d-415f-9747-57225d3247be None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:52.599889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc44ff1-426d-415f-9747-57225d3247be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:52.618126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-0fc44ff1-426d-415f-9747-57225d3247be None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:52.619286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0fc44ff1-426d-415f-9747-57225d3247be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:52.684816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 536/1079] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:52.707193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f49967-93a6-4d84-b335-0029d1cd75b6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:52.784967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:52.809159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f49967-93a6-4d84-b335-0029d1cd75b6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:52.842337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c81b69-1b16-4ef0-bddf-fc62cb404f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:52.842886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c81b69-1b16-4ef0-bddf-fc62cb404f2d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:52.843163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c81b69-1b16-4ef0-bddf-fc62cb404f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:52.843434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c81b69-1b16-4ef0-bddf-fc62cb404f2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:52.853877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c81b69-1b16-4ef0-bddf-fc62cb404f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d4c81b69-1b16-4ef0-bddf-fc62cb404f2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:52.858865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:52.859801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:52.860735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 544/1080] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:52.866724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44ff1-426d-415f-9747-57225d3247be None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:52.949400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:52.965304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc44ff1-426d-415f-9747-57225d3247be None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:52.997997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:52.998589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:52.998863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:52.999142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:52.999779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:53.000076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:53.001459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:53.009393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 537/1081] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:53.015028 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c81b69-1b16-4ef0-bddf-fc62cb404f2d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:53.117942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c81b69-1b16-4ef0-bddf-fc62cb404f2d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:53.150658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535dd49-59a4-4934-b435-3bb95039a22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:53.151287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535dd49-59a4-4934-b435-3bb95039a22c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:53.151564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535dd49-59a4-4934-b435-3bb95039a22c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:53.151836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535dd49-59a4-4934-b435-3bb95039a22c None admin] PATH_INFO: `/v3/users/608f1e7e96024330a184d7846fcdc902` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:53.152989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535dd49-59a4-4934-b435-3bb95039a22c None admin] RBAC: Authorizing `identity:get_user(user_id=608f1e7e96024330a184d7846fcdc902)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:53.154424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535dd49-59a4-4934-b435-3bb95039a22c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:53.156044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 545/1082] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:16:53 2022] GET /identity/v3/users/608f1e7e96024330a184d7846fcdc902 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:53.162789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:53.262753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f7c56-f579-4fdd-8ff9-ac6e5e33c127 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:53.295538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec48c1-9153-47e0-b794-1d85404417f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:53.296179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec48c1-9153-47e0-b794-1d85404417f9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:53.296458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec48c1-9153-47e0-b794-1d85404417f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:53.296723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec48c1-9153-47e0-b794-1d85404417f9 None admin] PATH_INFO: `/v3/projects/6d112dc320a14edebd79c3863e6d2f01` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:53.297818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec48c1-9153-47e0-b794-1d85404417f9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d112dc320a14edebd79c3863e6d2f01)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:53.299336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec48c1-9153-47e0-b794-1d85404417f9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:53.300978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 538/1083] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:16:53 2022] GET /identity/v3/projects/6d112dc320a14edebd79c3863e6d2f01 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:53.306931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535dd49-59a4-4934-b435-3bb95039a22c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:53.310754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:53.404193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535dd49-59a4-4934-b435-3bb95039a22c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:53.435645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56123ec0-0a43-4517-8894-19f761696f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:53.436314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56123ec0-0a43-4517-8894-19f761696f6e None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:53.436591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56123ec0-0a43-4517-8894-19f761696f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:53.436860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56123ec0-0a43-4517-8894-19f761696f6e None admin] PATH_INFO: `/v3/projects/6d112dc320a14edebd79c3863e6d2f01/users/608f1e7e96024330a184d7846fcdc902/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:53.438717 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56123ec0-0a43-4517-8894-19f761696f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=6d112dc320a14edebd79c3863e6d2f01, user_id=608f1e7e96024330a184d7846fcdc902, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:53.440786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56123ec0-0a43-4517-8894-19f761696f6e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:53.452935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 546/1084] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:16:53 2022] PUT /identity/v3/projects/6d112dc320a14edebd79c3863e6d2f01/users/608f1e7e96024330a184d7846fcdc902/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:16:54.132529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b40c-bca7-41f3-aa56-8b0407fedb0c None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:54.133074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b40c-bca7-41f3-aa56-8b0407fedb0c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:54.133189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7b40c-bca7-41f3-aa56-8b0407fedb0c None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:54.134522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 539/1085] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:54.139714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778d773-16cb-4016-aae8-ddcefacf5c4d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:54.140130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778d773-16cb-4016-aae8-ddcefacf5c4d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:54.140301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778d773-16cb-4016-aae8-ddcefacf5c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:54.158987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2778d773-16cb-4016-aae8-ddcefacf5c4d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:54.159881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2778d773-16cb-4016-aae8-ddcefacf5c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:54.218022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:54.227563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 547/1086] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:54.250880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7b40c-bca7-41f3-aa56-8b0407fedb0c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:54.269597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:54.353949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7b40c-bca7-41f3-aa56-8b0407fedb0c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:54.387035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db3cc4-95df-4c5d-9eb5-26046399a52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:54.387654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3cc4-95df-4c5d-9eb5-26046399a52d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:54.387932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3cc4-95df-4c5d-9eb5-26046399a52d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:54.388231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db3cc4-95df-4c5d-9eb5-26046399a52d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:54.398833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db3cc4-95df-4c5d-9eb5-26046399a52d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-10db3cc4-95df-4c5d-9eb5-26046399a52d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:54.403459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:54.409998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:54.410962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 540/1087] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Aug 11 16:16:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:54.411688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778d773-16cb-4016-aae8-ddcefacf5c4d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:54.507012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778d773-16cb-4016-aae8-ddcefacf5c4d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:54.541703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:54.542335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:54.542610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:54.542878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:54.543556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:54.543856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:54.545295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:54.554738 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 548/1088] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Aug 11 16:16:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:54.560551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db3cc4-95df-4c5d-9eb5-26046399a52d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:54.639571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:54.664284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db3cc4-95df-4c5d-9eb5-26046399a52d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:54.697815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:54.698434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:54.698710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:54.698977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:54.709651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:54.711083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:54.725948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:54.726905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:54.727818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:54.727818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:54.727818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:54.727818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 541/1089] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:54.732833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:54.788991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:54.833895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d1bee-12d5-4e63-8d52-0581f50ec39c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:54.867011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:54.867615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:54.867894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:54.868196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:54.868837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:54.869135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:54.870485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:54.883203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 549/1090] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:54.888894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:54.988610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81997-8606-4e1e-836e-1c4e07daccc0 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.014363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:55.022732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:55.023359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.023634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.023899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.031454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:55.032920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:16:55.047006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:55.048043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:55.049138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 542/1091] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Aug 11 16:16:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:16:55.054149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:55.152576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53cdc0-2ce6-42f1-aa54-a22cd1ff9664 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.169246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:55.187290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f84d44-72ba-4ae3-b34f-a4812088f0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:55.187894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f84d44-72ba-4ae3-b34f-a4812088f0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.188202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f84d44-72ba-4ae3-b34f-a4812088f0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.188474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f84d44-72ba-4ae3-b34f-a4812088f0a5 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.190003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f84d44-72ba-4ae3-b34f-a4812088f0a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:55.190309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f84d44-72ba-4ae3-b34f-a4812088f0a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:55.191892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f84d44-72ba-4ae3-b34f-a4812088f0a5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:55.209262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 550/1092] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Aug 11 16:16:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:55.215065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:55.313067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101e458-09e1-4cac-b702-514043b5d10e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.346336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d371411-3a47-4174-add8-b4bd93b9e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:55.346933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d371411-3a47-4174-add8-b4bd93b9e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.347212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d371411-3a47-4174-add8-b4bd93b9e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.347483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d371411-3a47-4174-add8-b4bd93b9e7f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.402083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:55.415422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d371411-3a47-4174-add8-b4bd93b9e7f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:55.416899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d371411-3a47-4174-add8-b4bd93b9e7f8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:55.437280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993007-8c01-4489-8704-ee17ed880cb3 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.437558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993007-8c01-4489-8704-ee17ed880cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.437828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993007-8c01-4489-8704-ee17ed880cb3 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.439496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 551/1093] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:16:55.444386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371292c-2b8e-4ddf-a318-5267ed433d37 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.444663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371292c-2b8e-4ddf-a318-5267ed433d37 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.444928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371292c-2b8e-4ddf-a318-5267ed433d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.463237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6371292c-2b8e-4ddf-a318-5267ed433d37 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:55.464454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6371292c-2b8e-4ddf-a318-5267ed433d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:55.516585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 543/1094] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:16:55.523070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d371411-3a47-4174-add8-b4bd93b9e7f8 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:55.530649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 552/1095] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:16:55.552575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371292c-2b8e-4ddf-a318-5267ed433d37 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:55.607530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:55.620706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d371411-3a47-4174-add8-b4bd93b9e7f8 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.655668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:55.656313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.656589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.656854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.657103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371292c-2b8e-4ddf-a318-5267ed433d37 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.660468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:55.667511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:55.668982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:16:55.683503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:55.684669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:16:55.685776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:16:55.685776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:16:55.685776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:55.685776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 544/1096] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:55.690499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:55.694050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26649d-5591-433e-a1eb-6f7108525d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:55.694686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26649d-5591-433e-a1eb-6f7108525d2a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.694965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26649d-5591-433e-a1eb-6f7108525d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.695238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26649d-5591-433e-a1eb-6f7108525d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.705261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26649d-5591-433e-a1eb-6f7108525d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ec26649d-5591-433e-a1eb-6f7108525d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:16:55.710141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:55.711058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:16:55.711928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 553/1097] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:16:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:55.717263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26649d-5591-433e-a1eb-6f7108525d2a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:55.788379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3eea3-19c0-4b2c-b4c1-63d01b23605e None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.813625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26649d-5591-433e-a1eb-6f7108525d2a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.821931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:55.822520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.822803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.823068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.823703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:55.824390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:55.825486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:55.833940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:55.837747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 545/1098] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:55.843250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:55.852910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:55.853511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.853784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.854051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.854709 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:55.855008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:55.856496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:55.869444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 554/1099] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:16:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:55.870384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:55.896863 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:55.943062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd97186-b9f5-465b-8717-64f931e6d113 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.969537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1681348-8870-42da-ba0c-1ca951475377 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:55.978278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:55.978904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:55.979180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:55.979449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:55.985889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:55.987591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:16:55.998677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:16:55.999583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:16:56.001177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 546/1100] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:16:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:16:56.002767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63f5fb-d38c-4774-92b3-90bb01d457f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:56.003392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63f5fb-d38c-4774-92b3-90bb01d457f5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:56.003670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63f5fb-d38c-4774-92b3-90bb01d457f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:56.003939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63f5fb-d38c-4774-92b3-90bb01d457f5 None admin] PATH_INFO: `/v3/users/608f1e7e96024330a184d7846fcdc902` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:56.005348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63f5fb-d38c-4774-92b3-90bb01d457f5 None admin] RBAC: Authorizing `identity:get_user(user_id=608f1e7e96024330a184d7846fcdc902)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:56.005520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:56.006754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63f5fb-d38c-4774-92b3-90bb01d457f5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:56.012872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 555/1101] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:16:55 2022] GET /identity/v3/users/608f1e7e96024330a184d7846fcdc902 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:56.013770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63f5fb-d38c-4774-92b3-90bb01d457f5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:56.065553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:56.107407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:56.107580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb389e-405c-41c5-8974-656ab026c408 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:56.114216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63f5fb-d38c-4774-92b3-90bb01d457f5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:56.140563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:56.141095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:56.141372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:56.141641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:56.142877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:56.143194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:56.144629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:56.147673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86ca5b-d789-4db0-974b-8fabee2d1f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:56.148299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86ca5b-d789-4db0-974b-8fabee2d1f08 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:56.148577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86ca5b-d789-4db0-974b-8fabee2d1f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:56.148845 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86ca5b-d789-4db0-974b-8fabee2d1f08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:56.156465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 547/1102] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:16:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:56.163334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:56.215419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86ca5b-d789-4db0-974b-8fabee2d1f08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:56.216883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86ca5b-d789-4db0-974b-8fabee2d1f08 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:56.260355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdb49a-ea43-4199-8fca-aa80c6c48680 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:56.280502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:56.295261 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:56.295846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:56.296147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:56.296411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:56.297123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] RBAC: Adding query filter params (role.id=34515f0849514506bafe075b65e3aa5e, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=2e3819e78802469c83c09a1cdd865f58) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:56.297420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:56.299115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:56.304661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 548/1103] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:56 2022] GET /identity/v3/role_assignments?user.id=2e3819e78802469c83c09a1cdd865f58&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=34515f0849514506bafe075b65e3aa5e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:16:56.307129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:56.316402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 556/1104] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:16:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:16:56.322451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:56.428580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc59bb-de00-47b2-8a45-83d093bee083 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:56.464068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f4bd7-64ed-413a-a530-8f9ed1e20a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:56.464725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f4bd7-64ed-413a-a530-8f9ed1e20a79 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:56.465002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f4bd7-64ed-413a-a530-8f9ed1e20a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:56.465274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f4bd7-64ed-413a-a530-8f9ed1e20a79 None admin] PATH_INFO: `/v3/projects/6d112dc320a14edebd79c3863e6d2f01` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:56.466407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f4bd7-64ed-413a-a530-8f9ed1e20a79 None admin] RBAC: Authorizing `identity:get_project(project_id=6d112dc320a14edebd79c3863e6d2f01)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:56.467833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f4bd7-64ed-413a-a530-8f9ed1e20a79 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:56.469500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 549/1105] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:16:56 2022] GET /identity/v3/projects/6d112dc320a14edebd79c3863e6d2f01 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:56.476506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86ca5b-d789-4db0-974b-8fabee2d1f08 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:56.585310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86ca5b-d789-4db0-974b-8fabee2d1f08 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:56.620367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a4c0-bad1-4367-903b-d0f125b01f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:56.620975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a4c0-bad1-4367-903b-d0f125b01f42 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:56.621254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a4c0-bad1-4367-903b-d0f125b01f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:56.621519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a4c0-bad1-4367-903b-d0f125b01f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:56.622243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a4c0-bad1-4367-903b-d0f125b01f42 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=6d112dc320a14edebd79c3863e6d2f01, user.id=608f1e7e96024330a184d7846fcdc902) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:16:56.622546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a4c0-bad1-4367-903b-d0f125b01f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:56.623968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a4c0-bad1-4367-903b-d0f125b01f42 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:56.631165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 557/1106] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:16:56 2022] GET /identity/v3/role_assignments?user.id=608f1e7e96024330a184d7846fcdc902&scope.project.id=6d112dc320a14edebd79c3863e6d2f01&role.id=29fd031edd2b4f65aed8942853150f92 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:58.657401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479734e7-c322-4ecb-8d5c-c6ed78e639c1 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:58.658443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479734e7-c322-4ecb-8d5c-c6ed78e639c1 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:58.658631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479734e7-c322-4ecb-8d5c-c6ed78e639c1 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:58.659987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 550/1107] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:16:58.665143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5869b89-528d-4af1-a2d8-333e2aca2813 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:58.665574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5869b89-528d-4af1-a2d8-333e2aca2813 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:58.665695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5869b89-528d-4af1-a2d8-333e2aca2813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:58.683826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f5869b89-528d-4af1-a2d8-333e2aca2813 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:16:58.685031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f5869b89-528d-4af1-a2d8-333e2aca2813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:16:58.735001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:58.748882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 558/1108] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:16:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:16:58.772711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479734e7-c322-4ecb-8d5c-c6ed78e639c1 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:58.825585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:16:58.872883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479734e7-c322-4ecb-8d5c-c6ed78e639c1 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:58.905832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abfb63-7f89-4d86-a621-bcf4a45a3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:58.906446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abfb63-7f89-4d86-a621-bcf4a45a3076 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:58.906726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abfb63-7f89-4d86-a621-bcf4a45a3076 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:58.906999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abfb63-7f89-4d86-a621-bcf4a45a3076 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:58.908873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abfb63-7f89-4d86-a621-bcf4a45a3076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:58.910252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abfb63-7f89-4d86-a621-bcf4a45a3076 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:58.911793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 551/1109] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:16:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:16:58.917491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5869b89-528d-4af1-a2d8-333e2aca2813 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:16:59.011930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5869b89-528d-4af1-a2d8-333e2aca2813 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:16:59.045041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917473ba-be0b-40f0-b482-37ada57b9829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:16:59.045639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917473ba-be0b-40f0-b482-37ada57b9829 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:16:59.045913 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917473ba-be0b-40f0-b482-37ada57b9829 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:16:59.046180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917473ba-be0b-40f0-b482-37ada57b9829 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:16:59.046825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917473ba-be0b-40f0-b482-37ada57b9829 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:16:59.048263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917473ba-be0b-40f0-b482-37ada57b9829 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:16:59.068652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 559/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:16:58 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:17:01.246429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558aa12-bdbc-443d-96b8-20639860b508 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:01.246987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558aa12-bdbc-443d-96b8-20639860b508 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:01.246987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558aa12-bdbc-443d-96b8-20639860b508 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:01.248306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 552/1111] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:01.254402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94329094-547d-4ab5-8090-dbb7c10a9385 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:01.254782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94329094-547d-4ab5-8090-dbb7c10a9385 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:01.254975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94329094-547d-4ab5-8090-dbb7c10a9385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:01.272608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-94329094-547d-4ab5-8090-dbb7c10a9385 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:01.273778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-94329094-547d-4ab5-8090-dbb7c10a9385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:01.312019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:01.338421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 560/1112] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:01.361533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558aa12-bdbc-443d-96b8-20639860b508 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:01.458906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558aa12-bdbc-443d-96b8-20639860b508 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:01.478011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:01.492321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3157-7ff7-453b-8f5d-e0763364e56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:01.492933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3157-7ff7-453b-8f5d-e0763364e56a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:01.493207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3157-7ff7-453b-8f5d-e0763364e56a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:01.493473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3157-7ff7-453b-8f5d-e0763364e56a None admin] PATH_INFO: `/v3/roles/3b5810bc1bb047f7851cef3afb74dcd0` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:01.494854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3157-7ff7-453b-8f5d-e0763364e56a None admin] RBAC: Authorizing `identity:get_role(role_id=3b5810bc1bb047f7851cef3afb74dcd0)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:01.496225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3157-7ff7-453b-8f5d-e0763364e56a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:01.497515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 553/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:01 2022] GET /identity/v3/roles/3b5810bc1bb047f7851cef3afb74dcd0 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:01.503954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94329094-547d-4ab5-8090-dbb7c10a9385 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:01.599422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94329094-547d-4ab5-8090-dbb7c10a9385 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:01.631581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ae253-85ad-48b4-abb0-7080dc6ede04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:01.632222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ae253-85ad-48b4-abb0-7080dc6ede04 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:01.632497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ae253-85ad-48b4-abb0-7080dc6ede04 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:01.632765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ae253-85ad-48b4-abb0-7080dc6ede04 None admin] PATH_INFO: `/v3/users/e8c810f6ce3148488748c0c6537bb8e7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:01.645561 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ae253-85ad-48b4-abb0-7080dc6ede04 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c810f6ce3148488748c0c6537bb8e7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:01.646965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ae253-85ad-48b4-abb0-7080dc6ede04 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:01.648573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 561/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:01 2022] GET /identity/v3/users/e8c810f6ce3148488748c0c6537bb8e7 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:01.654992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3157-7ff7-453b-8f5d-e0763364e56a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:01.750327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3157-7ff7-453b-8f5d-e0763364e56a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:01.782593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7f29-f9e1-43ad-90bf-3195453a3dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:01.783872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7f29-f9e1-43ad-90bf-3195453a3dad None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:01.784175 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7f29-f9e1-43ad-90bf-3195453a3dad None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:01.784444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac7f29-f9e1-43ad-90bf-3195453a3dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:01.836548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:01.849725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac7f29-f9e1-43ad-90bf-3195453a3dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:01.851099 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac7f29-f9e1-43ad-90bf-3195453a3dad None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:01.945597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 554/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:17:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:17:01.951483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ae253-85ad-48b4-abb0-7080dc6ede04 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:01.982159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:02.047853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ae253-85ad-48b4-abb0-7080dc6ede04 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:02.083447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779b98a-e8c0-4bae-871a-874399fef844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:02.084211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779b98a-e8c0-4bae-871a-874399fef844 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:02.084556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779b98a-e8c0-4bae-871a-874399fef844 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:02.084917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779b98a-e8c0-4bae-871a-874399fef844 None admin] PATH_INFO: `/v3/projects/6d112dc320a14edebd79c3863e6d2f01` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:02.086104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779b98a-e8c0-4bae-871a-874399fef844 None admin] RBAC: Authorizing `identity:get_project(project_id=6d112dc320a14edebd79c3863e6d2f01)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:02.087584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779b98a-e8c0-4bae-871a-874399fef844 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:02.089457 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 562/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:01 2022] GET /identity/v3/projects/6d112dc320a14edebd79c3863e6d2f01 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:02.095990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7f29-f9e1-43ad-90bf-3195453a3dad None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:02.188793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:02.202306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac7f29-f9e1-43ad-90bf-3195453a3dad None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:02.238570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93295f-b614-4ec0-b91f-8bffbbaef5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:02.239176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93295f-b614-4ec0-b91f-8bffbbaef5db None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:02.239449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93295f-b614-4ec0-b91f-8bffbbaef5db None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:02.239714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93295f-b614-4ec0-b91f-8bffbbaef5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:02.240471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93295f-b614-4ec0-b91f-8bffbbaef5db None admin] RBAC: Adding query filter params (role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=6d112dc320a14edebd79c3863e6d2f01, user.id=e8c810f6ce3148488748c0c6537bb8e7) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:02.240778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93295f-b614-4ec0-b91f-8bffbbaef5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:02.242194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93295f-b614-4ec0-b91f-8bffbbaef5db None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:02.247783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 555/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:17:02 2022] GET /identity/v3/role_assignments?user.id=e8c810f6ce3148488748c0c6537bb8e7&scope.project.id=6d112dc320a14edebd79c3863e6d2f01&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:04.286707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed7735-2300-498e-99d9-9aeec28ecdc9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:04.287288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed7735-2300-498e-99d9-9aeec28ecdc9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:04.287288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed7735-2300-498e-99d9-9aeec28ecdc9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:04.288610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 563/1118] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:04.293792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436a2f-0b04-46ce-b66b-be3d334c5b46 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:04.294177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436a2f-0b04-46ce-b66b-be3d334c5b46 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:04.294330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436a2f-0b04-46ce-b66b-be3d334c5b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:04.312985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1e436a2f-0b04-46ce-b66b-be3d334c5b46 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:04.315475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1e436a2f-0b04-46ce-b66b-be3d334c5b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:04.379872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 556/1119] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:04.402266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed7735-2300-498e-99d9-9aeec28ecdc9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:04.499309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:04.504240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed7735-2300-498e-99d9-9aeec28ecdc9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:04.538911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f40c33-2f71-439e-a39e-1fecb0bef01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:04.539806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f40c33-2f71-439e-a39e-1fecb0bef01e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:04.539806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f40c33-2f71-439e-a39e-1fecb0bef01e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:04.540073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f40c33-2f71-439e-a39e-1fecb0bef01e None admin] PATH_INFO: `/v3/roles/3b5810bc1bb047f7851cef3afb74dcd0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:04.541492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f40c33-2f71-439e-a39e-1fecb0bef01e None admin] RBAC: Authorizing `identity:get_role(role_id=3b5810bc1bb047f7851cef3afb74dcd0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:04.542849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f40c33-2f71-439e-a39e-1fecb0bef01e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:04.544193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 564/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:04 2022] GET /identity/v3/roles/3b5810bc1bb047f7851cef3afb74dcd0 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:04.551116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436a2f-0b04-46ce-b66b-be3d334c5b46 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:04.628589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:04.646509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436a2f-0b04-46ce-b66b-be3d334c5b46 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:04.679243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3736b1-fcb0-4334-acf9-1b4e24abb0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:04.679854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3736b1-fcb0-4334-acf9-1b4e24abb0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:04.680148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3736b1-fcb0-4334-acf9-1b4e24abb0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:04.680418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3736b1-fcb0-4334-acf9-1b4e24abb0f8 None admin] PATH_INFO: `/v3/users/e8c810f6ce3148488748c0c6537bb8e7` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:04.681525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3736b1-fcb0-4334-acf9-1b4e24abb0f8 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c810f6ce3148488748c0c6537bb8e7)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:04.682938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3736b1-fcb0-4334-acf9-1b4e24abb0f8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:04.684506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 557/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:04 2022] GET /identity/v3/users/e8c810f6ce3148488748c0c6537bb8e7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:04.690882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f40c33-2f71-439e-a39e-1fecb0bef01e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:04.787281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f40c33-2f71-439e-a39e-1fecb0bef01e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:04.819860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bbc13-a6b9-4a40-a8c8-5a77bf627672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:04.820507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bbc13-a6b9-4a40-a8c8-5a77bf627672 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:04.820789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bbc13-a6b9-4a40-a8c8-5a77bf627672 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:04.821061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bbc13-a6b9-4a40-a8c8-5a77bf627672 None admin] PATH_INFO: `/v3/projects/6d112dc320a14edebd79c3863e6d2f01` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:04.822171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bbc13-a6b9-4a40-a8c8-5a77bf627672 None admin] RBAC: Authorizing `identity:get_project(project_id=6d112dc320a14edebd79c3863e6d2f01)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:04.823584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bbc13-a6b9-4a40-a8c8-5a77bf627672 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:04.825219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 565/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:04 2022] GET /identity/v3/projects/6d112dc320a14edebd79c3863e6d2f01 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:04.831134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3736b1-fcb0-4334-acf9-1b4e24abb0f8 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:04.927384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3736b1-fcb0-4334-acf9-1b4e24abb0f8 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:04.960028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6d307-b9a9-4958-9f88-9591ec504b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:04.960758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d307-b9a9-4958-9f88-9591ec504b40 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:04.961030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d307-b9a9-4958-9f88-9591ec504b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:04.961294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6d307-b9a9-4958-9f88-9591ec504b40 None admin] PATH_INFO: `/v3/projects/6d112dc320a14edebd79c3863e6d2f01/users/e8c810f6ce3148488748c0c6537bb8e7/roles/3b5810bc1bb047f7851cef3afb74dcd0` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:04.963375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6d307-b9a9-4958-9f88-9591ec504b40 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d112dc320a14edebd79c3863e6d2f01, user_id=e8c810f6ce3148488748c0c6537bb8e7, role_id=3b5810bc1bb047f7851cef3afb74dcd0)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:04.964807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6d307-b9a9-4958-9f88-9591ec504b40 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:04.977918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 558/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:17:04 2022] PUT /identity/v3/projects/6d112dc320a14edebd79c3863e6d2f01/users/e8c810f6ce3148488748c0c6537bb8e7/roles/3b5810bc1bb047f7851cef3afb74dcd0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:17:06.931017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6640-67eb-4a09-804f-15590d6b2787 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:06.931548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6640-67eb-4a09-804f-15590d6b2787 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:06.931548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6640-67eb-4a09-804f-15590d6b2787 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:06.932908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 566/1124] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:06.937868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58ed2-1c39-435d-b81a-873646d51a0b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:06.938251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58ed2-1c39-435d-b81a-873646d51a0b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:06.938416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58ed2-1c39-435d-b81a-873646d51a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:06.957442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1da58ed2-1c39-435d-b81a-873646d51a0b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:06.958625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1da58ed2-1c39-435d-b81a-873646d51a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:06.961384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:07.025121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 559/1125] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:07.047710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6640-67eb-4a09-804f-15590d6b2787 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:07.085377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:07.148036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6640-67eb-4a09-804f-15590d6b2787 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:07.180184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5034446-9fd2-4f63-82eb-fdbd629943fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:07.180784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5034446-9fd2-4f63-82eb-fdbd629943fd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:07.181108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5034446-9fd2-4f63-82eb-fdbd629943fd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:07.181328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5034446-9fd2-4f63-82eb-fdbd629943fd None admin] PATH_INFO: `/v3/roles/3b5810bc1bb047f7851cef3afb74dcd0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:07.182686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5034446-9fd2-4f63-82eb-fdbd629943fd None admin] RBAC: Authorizing `identity:get_role(role_id=3b5810bc1bb047f7851cef3afb74dcd0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:07.184032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5034446-9fd2-4f63-82eb-fdbd629943fd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:07.185354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 567/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:07 2022] GET /identity/v3/roles/3b5810bc1bb047f7851cef3afb74dcd0 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:07.191855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da58ed2-1c39-435d-b81a-873646d51a0b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:07.286919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da58ed2-1c39-435d-b81a-873646d51a0b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:07.320322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207a203-c637-4d70-a674-ce97134761b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:07.320938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207a203-c637-4d70-a674-ce97134761b7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:07.321217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207a203-c637-4d70-a674-ce97134761b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:07.321490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207a203-c637-4d70-a674-ce97134761b7 None admin] PATH_INFO: `/v3/users/e8c810f6ce3148488748c0c6537bb8e7` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:07.322613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207a203-c637-4d70-a674-ce97134761b7 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c810f6ce3148488748c0c6537bb8e7)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:07.324024 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207a203-c637-4d70-a674-ce97134761b7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:07.325607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 560/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:07 2022] GET /identity/v3/users/e8c810f6ce3148488748c0c6537bb8e7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:07.331734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5034446-9fd2-4f63-82eb-fdbd629943fd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:07.429446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5034446-9fd2-4f63-82eb-fdbd629943fd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:07.431740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:07.462351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efd316-d912-4a35-b6ad-919c4ab9bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:07.462943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd316-d912-4a35-b6ad-919c4ab9bf2e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:07.463223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd316-d912-4a35-b6ad-919c4ab9bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:07.463492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efd316-d912-4a35-b6ad-919c4ab9bf2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:07.528735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efd316-d912-4a35-b6ad-919c4ab9bf2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:07.530157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efd316-d912-4a35-b6ad-919c4ab9bf2e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:07.622849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 568/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:17:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:17:07.629295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207a203-c637-4d70-a674-ce97134761b7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:07.639738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:07.732184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207a203-c637-4d70-a674-ce97134761b7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:07.764203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109c9ca-b6ba-490b-b481-7832e0fc0470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:07.764812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c9ca-b6ba-490b-b481-7832e0fc0470 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:07.765085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c9ca-b6ba-490b-b481-7832e0fc0470 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:07.765351 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109c9ca-b6ba-490b-b481-7832e0fc0470 None admin] PATH_INFO: `/v3/projects/6d112dc320a14edebd79c3863e6d2f01` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:07.766443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109c9ca-b6ba-490b-b481-7832e0fc0470 None admin] RBAC: Authorizing `identity:get_project(project_id=6d112dc320a14edebd79c3863e6d2f01)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:07.768629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109c9ca-b6ba-490b-b481-7832e0fc0470 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:07.770216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 561/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:07 2022] GET /identity/v3/projects/6d112dc320a14edebd79c3863e6d2f01 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:07.776293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efd316-d912-4a35-b6ad-919c4ab9bf2e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:07.779305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:07.871990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efd316-d912-4a35-b6ad-919c4ab9bf2e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:07.904977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0e85f-4552-4b5b-b84a-68a2d30779ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:07.905571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0e85f-4552-4b5b-b84a-68a2d30779ad None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:07.905855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0e85f-4552-4b5b-b84a-68a2d30779ad None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:07.906123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0e85f-4552-4b5b-b84a-68a2d30779ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:07.906835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0e85f-4552-4b5b-b84a-68a2d30779ad None admin] RBAC: Adding query filter params (role.id=3b5810bc1bb047f7851cef3afb74dcd0, scope.project.id=6d112dc320a14edebd79c3863e6d2f01, user.id=e8c810f6ce3148488748c0c6537bb8e7) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:07.907143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0e85f-4552-4b5b-b84a-68a2d30779ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:07.908848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0e85f-4552-4b5b-b84a-68a2d30779ad None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:07.914431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 569/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:17:07 2022] GET /identity/v3/role_assignments?user.id=e8c810f6ce3148488748c0c6537bb8e7&scope.project.id=6d112dc320a14edebd79c3863e6d2f01&role.id=3b5810bc1bb047f7851cef3afb74dcd0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:09.937312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f3dd4-6671-4271-befa-97030ebf4dc5 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:09.937870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f3dd4-6671-4271-befa-97030ebf4dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:09.937870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f3dd4-6671-4271-befa-97030ebf4dc5 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:09.939193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 562/1131] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:09.944695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a835-eb69-4aaf-a995-b5a02e2925d2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:09.945081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a835-eb69-4aaf-a995-b5a02e2925d2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:09.945250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a835-eb69-4aaf-a995-b5a02e2925d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:09.962767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-97b1a835-eb69-4aaf-a995-b5a02e2925d2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:09.963954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-97b1a835-eb69-4aaf-a995-b5a02e2925d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:10.023704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:10.029456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 570/1132] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:10.053775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f3dd4-6671-4271-befa-97030ebf4dc5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:10.129721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:10.153260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f3dd4-6671-4271-befa-97030ebf4dc5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:10.186142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf064a51-772c-4a5f-b76f-1a4cdcdbc13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:10.186756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf064a51-772c-4a5f-b76f-1a4cdcdbc13e None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:10.187035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf064a51-772c-4a5f-b76f-1a4cdcdbc13e None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:10.187308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf064a51-772c-4a5f-b76f-1a4cdcdbc13e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:10.188496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf064a51-772c-4a5f-b76f-1a4cdcdbc13e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:10.190277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf064a51-772c-4a5f-b76f-1a4cdcdbc13e None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:10.191833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 563/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:17:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:10.197689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1a835-eb69-4aaf-a995-b5a02e2925d2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:10.293595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1a835-eb69-4aaf-a995-b5a02e2925d2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:10.326239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238c2ce-c47d-44d4-8d54-6092ef2e83ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:10.326840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c2ce-c47d-44d4-8d54-6092ef2e83ec None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:10.327118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c2ce-c47d-44d4-8d54-6092ef2e83ec None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:10.327390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238c2ce-c47d-44d4-8d54-6092ef2e83ec None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:10.328027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238c2ce-c47d-44d4-8d54-6092ef2e83ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:10.329998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238c2ce-c47d-44d4-8d54-6092ef2e83ec None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:10.355928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 571/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:17:10 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:17:12.383888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b042d-2857-4f8f-b038-eea8dd39f821 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:12.384432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b042d-2857-4f8f-b038-eea8dd39f821 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:12.384432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b042d-2857-4f8f-b038-eea8dd39f821 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:12.385711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 564/1135] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:12.390874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b094be5-6763-4369-8c83-c8c099b2404e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:12.391305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b094be5-6763-4369-8c83-c8c099b2404e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:12.391427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b094be5-6763-4369-8c83-c8c099b2404e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:12.410195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7b094be5-6763-4369-8c83-c8c099b2404e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:12.411376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7b094be5-6763-4369-8c83-c8c099b2404e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:12.456773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:12.478601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 572/1136] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:12.502729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b042d-2857-4f8f-b038-eea8dd39f821 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:12.602465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b042d-2857-4f8f-b038-eea8dd39f821 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:12.635795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322afcd-a5ff-4a03-aa71-a66f995bddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:12.636380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322afcd-a5ff-4a03-aa71-a66f995bddb4 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:12.636660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322afcd-a5ff-4a03-aa71-a66f995bddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:12.636934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322afcd-a5ff-4a03-aa71-a66f995bddb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:12.638044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322afcd-a5ff-4a03-aa71-a66f995bddb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:12.639474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322afcd-a5ff-4a03-aa71-a66f995bddb4 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:12.641059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 565/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:17:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:12.647439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b094be5-6763-4369-8c83-c8c099b2404e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:12.743276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b094be5-6763-4369-8c83-c8c099b2404e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:12.776967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f64214-767c-44ed-8693-3fbebd9407f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:12.777562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f64214-767c-44ed-8693-3fbebd9407f3 None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:12.777843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f64214-767c-44ed-8693-3fbebd9407f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:12.778115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f64214-767c-44ed-8693-3fbebd9407f3 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:12.778747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f64214-767c-44ed-8693-3fbebd9407f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:12.780186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f64214-767c-44ed-8693-3fbebd9407f3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:12.800458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 573/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:17:12 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:17:14.843265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e96d31-313d-4168-9e75-1ed949dc10bc None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:14.843821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e96d31-313d-4168-9e75-1ed949dc10bc None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:14.843821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e96d31-313d-4168-9e75-1ed949dc10bc None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:14.845393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 566/1139] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:14.850404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f09ee-ec03-44d5-bf8c-8ad5d4890a00 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:14.850828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f09ee-ec03-44d5-bf8c-8ad5d4890a00 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:14.850952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f09ee-ec03-44d5-bf8c-8ad5d4890a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:14.868921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b01f09ee-ec03-44d5-bf8c-8ad5d4890a00 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:14.870084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b01f09ee-ec03-44d5-bf8c-8ad5d4890a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:14.898089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:14.934508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 574/1140] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:14.957888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e96d31-313d-4168-9e75-1ed949dc10bc None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:14.972632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:15.058006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e96d31-313d-4168-9e75-1ed949dc10bc None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:15.090923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f973-a3a3-4dfb-bb33-d22f8acf0798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:15.091534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f973-a3a3-4dfb-bb33-d22f8acf0798 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:15.091813 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f973-a3a3-4dfb-bb33-d22f8acf0798 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:15.092081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81f973-a3a3-4dfb-bb33-d22f8acf0798 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:15.102620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81f973-a3a3-4dfb-bb33-d22f8acf0798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-da81f973-a3a3-4dfb-bb33-d22f8acf0798 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:17:15.107295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:17:15.108602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:17:15.109862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 567/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:17:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:17:15.114985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f09ee-ec03-44d5-bf8c-8ad5d4890a00 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:15.210920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f09ee-ec03-44d5-bf8c-8ad5d4890a00 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:15.244489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:15.245091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:15.245370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:15.245642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:15.246295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:15.246593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:15.247940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:15.255884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 575/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:17:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:15.261606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f973-a3a3-4dfb-bb33-d22f8acf0798 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:15.330447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:15.358817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81f973-a3a3-4dfb-bb33-d22f8acf0798 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:15.391519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42bc68-2a41-4716-84d9-883f48003cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:15.392670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42bc68-2a41-4716-84d9-883f48003cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:15.393126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42bc68-2a41-4716-84d9-883f48003cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:15.393403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc42bc68-2a41-4716-84d9-883f48003cd7 None admin] PATH_INFO: `/v3/users/22cc2f4064a14ae4985851ddb0c31dec` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:15.405928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42bc68-2a41-4716-84d9-883f48003cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=22cc2f4064a14ae4985851ddb0c31dec)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:15.407390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc42bc68-2a41-4716-84d9-883f48003cd7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:15.409701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 568/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:15 2022] GET /identity/v3/users/22cc2f4064a14ae4985851ddb0c31dec => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:15.415512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:15.443712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:15.512960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6590ed-fe22-48b7-9fbc-b5db88678914 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:15.545677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c47dd-c317-46bc-af1c-d9ce25bf998f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:15.546276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c47dd-c317-46bc-af1c-d9ce25bf998f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:15.546555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c47dd-c317-46bc-af1c-d9ce25bf998f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:15.546829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c47dd-c317-46bc-af1c-d9ce25bf998f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:15.611790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c47dd-c317-46bc-af1c-d9ce25bf998f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:15.613195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c47dd-c317-46bc-af1c-d9ce25bf998f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:15.640958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:15.711413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 576/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:17:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:17:15.717245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42bc68-2a41-4716-84d9-883f48003cd7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:15.815387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc42bc68-2a41-4716-84d9-883f48003cd7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:15.843425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:15.849699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09387d52-7f25-4409-bc96-a4f095e4144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:15.850315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09387d52-7f25-4409-bc96-a4f095e4144b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:15.850593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09387d52-7f25-4409-bc96-a4f095e4144b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:15.850866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09387d52-7f25-4409-bc96-a4f095e4144b None admin] PATH_INFO: `/v3/projects/030e1cdc9c5747beb3da263345ae463d` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:15.852001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09387d52-7f25-4409-bc96-a4f095e4144b None admin] RBAC: Authorizing `identity:get_project(project_id=030e1cdc9c5747beb3da263345ae463d)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:15.853456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09387d52-7f25-4409-bc96-a4f095e4144b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:15.855060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 569/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:15 2022] GET /identity/v3/projects/030e1cdc9c5747beb3da263345ae463d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:15.861950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c47dd-c317-46bc-af1c-d9ce25bf998f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:15.957474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c47dd-c317-46bc-af1c-d9ce25bf998f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:15.982036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:15.990588 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde1c6b-0b91-45e8-b9cc-f4c5e79c2365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:15.991178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde1c6b-0b91-45e8-b9cc-f4c5e79c2365 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:15.991452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde1c6b-0b91-45e8-b9cc-f4c5e79c2365 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:15.991715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde1c6b-0b91-45e8-b9cc-f4c5e79c2365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:15.992463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde1c6b-0b91-45e8-b9cc-f4c5e79c2365 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=030e1cdc9c5747beb3da263345ae463d, user.id=22cc2f4064a14ae4985851ddb0c31dec) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:15.992762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde1c6b-0b91-45e8-b9cc-f4c5e79c2365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:15.994151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde1c6b-0b91-45e8-b9cc-f4c5e79c2365 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:15.999268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 577/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:17:15 2022] GET /identity/v3/role_assignments?user.id=22cc2f4064a14ae4985851ddb0c31dec&scope.project.id=030e1cdc9c5747beb3da263345ae463d&role.id=29fd031edd2b4f65aed8942853150f92 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:17.956947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be22f9-8558-4483-b206-93954a9d0d0a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:17.957507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be22f9-8558-4483-b206-93954a9d0d0a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:17.957507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be22f9-8558-4483-b206-93954a9d0d0a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:17.958835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 570/1147] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:17.963947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a998b-01f6-4cc1-bdea-a1525b98ed74 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:17.964343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a998b-01f6-4cc1-bdea-a1525b98ed74 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:17.964537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a998b-01f6-4cc1-bdea-a1525b98ed74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:17.982440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c98a998b-01f6-4cc1-bdea-a1525b98ed74 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:17.983612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c98a998b-01f6-4cc1-bdea-a1525b98ed74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:18.047919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 578/1148] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:18.070557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be22f9-8558-4483-b206-93954a9d0d0a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:18.170396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be22f9-8558-4483-b206-93954a9d0d0a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:18.203664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6473f4-f81a-4d5d-9588-badfb8ea4811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:18.205016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6473f4-f81a-4d5d-9588-badfb8ea4811 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:18.205387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6473f4-f81a-4d5d-9588-badfb8ea4811 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:18.205671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6473f4-f81a-4d5d-9588-badfb8ea4811 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:18.215542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6473f4-f81a-4d5d-9588-badfb8ea4811 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-4c6473f4-f81a-4d5d-9588-badfb8ea4811 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:17:18.220276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:17:18.221388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:17:18.222301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 571/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:17:18.227355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a998b-01f6-4cc1-bdea-a1525b98ed74 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:18.322547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a998b-01f6-4cc1-bdea-a1525b98ed74 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:18.344734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:18.356592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:18.357185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:18.357458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:18.357724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:18.358375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:18.358669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:18.360013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:18.368699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 579/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:17:18.374656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6473f4-f81a-4d5d-9588-badfb8ea4811 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:18.433142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:18.474078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6473f4-f81a-4d5d-9588-badfb8ea4811 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:18.507149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19f48c-4ea4-41c7-8c3f-5a411c9468eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:18.507754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19f48c-4ea4-41c7-8c3f-5a411c9468eb None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:18.508031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19f48c-4ea4-41c7-8c3f-5a411c9468eb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:18.508333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19f48c-4ea4-41c7-8c3f-5a411c9468eb None admin] PATH_INFO: `/v3/users/22cc2f4064a14ae4985851ddb0c31dec` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:18.509444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19f48c-4ea4-41c7-8c3f-5a411c9468eb None admin] RBAC: Authorizing `identity:get_user(user_id=22cc2f4064a14ae4985851ddb0c31dec)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:18.510860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19f48c-4ea4-41c7-8c3f-5a411c9468eb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:18.512460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 572/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:18 2022] GET /identity/v3/users/22cc2f4064a14ae4985851ddb0c31dec => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:18.518183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:18.620319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ed838-37fd-4324-bcbb-72f2d3cc6c6a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:18.652684 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99ff63-1c93-4c04-bc93-c65da97d301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:18.653322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99ff63-1c93-4c04-bc93-c65da97d301e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:18.653597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99ff63-1c93-4c04-bc93-c65da97d301e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:18.653861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99ff63-1c93-4c04-bc93-c65da97d301e None admin] PATH_INFO: `/v3/projects/030e1cdc9c5747beb3da263345ae463d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:18.654955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99ff63-1c93-4c04-bc93-c65da97d301e None admin] RBAC: Authorizing `identity:get_project(project_id=030e1cdc9c5747beb3da263345ae463d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:18.656374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99ff63-1c93-4c04-bc93-c65da97d301e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:18.658144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 580/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:18 2022] GET /identity/v3/projects/030e1cdc9c5747beb3da263345ae463d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:18.664273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19f48c-4ea4-41c7-8c3f-5a411c9468eb None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:18.760846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19f48c-4ea4-41c7-8c3f-5a411c9468eb None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:18.785694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:18.794182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3a698-b23c-4fa9-9d3c-7ce89fcbca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:18.794823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3a698-b23c-4fa9-9d3c-7ce89fcbca7a None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:18.795101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3a698-b23c-4fa9-9d3c-7ce89fcbca7a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:18.795374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3a698-b23c-4fa9-9d3c-7ce89fcbca7a None admin] PATH_INFO: `/v3/projects/030e1cdc9c5747beb3da263345ae463d/users/22cc2f4064a14ae4985851ddb0c31dec/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:18.797245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3a698-b23c-4fa9-9d3c-7ce89fcbca7a None admin] RBAC: Authorizing `identity:create_grant(project_id=030e1cdc9c5747beb3da263345ae463d, user_id=22cc2f4064a14ae4985851ddb0c31dec, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:18.799241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3a698-b23c-4fa9-9d3c-7ce89fcbca7a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:18.811987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 573/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:17:18 2022] PUT /identity/v3/projects/030e1cdc9c5747beb3da263345ae463d/users/22cc2f4064a14ae4985851ddb0c31dec/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:17:20.788139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bac14-870d-48d7-9f1e-768121e306e1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:20.788676 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bac14-870d-48d7-9f1e-768121e306e1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:20.788676 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bac14-870d-48d7-9f1e-768121e306e1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:20.790008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 581/1154] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:20.795133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966a76a-00f3-4f37-9273-f660a5fbc9cd None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:20.795610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966a76a-00f3-4f37-9273-f660a5fbc9cd None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:20.795702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966a76a-00f3-4f37-9273-f660a5fbc9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:20.814434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e966a76a-00f3-4f37-9273-f660a5fbc9cd None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:20.815864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e966a76a-00f3-4f37-9273-f660a5fbc9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:20.880196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 574/1155] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:20.903246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bac14-870d-48d7-9f1e-768121e306e1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:20.956363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:21.003388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bac14-870d-48d7-9f1e-768121e306e1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:21.036039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1bb04-e3e6-41bb-ac7c-d4ee02df2a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:21.036684 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1bb04-e3e6-41bb-ac7c-d4ee02df2a36 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:21.036963 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1bb04-e3e6-41bb-ac7c-d4ee02df2a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:21.037234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1bb04-e3e6-41bb-ac7c-d4ee02df2a36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:21.047595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1bb04-e3e6-41bb-ac7c-d4ee02df2a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-37f1bb04-e3e6-41bb-ac7c-d4ee02df2a36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:17:21.052885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:17:21.054030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:17:21.054891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 582/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:17:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:17:21.060158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966a76a-00f3-4f37-9273-f660a5fbc9cd None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:21.156672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966a76a-00f3-4f37-9273-f660a5fbc9cd None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:21.174254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:21.193354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:21.193946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:21.194218 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:21.194482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:21.195132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:21.195428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:21.196806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:21.205255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 575/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:17:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:21.212354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1bb04-e3e6-41bb-ac7c-d4ee02df2a36 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:21.310246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1bb04-e3e6-41bb-ac7c-d4ee02df2a36 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:21.323466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:21.343829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6431c-4e4a-4d0c-9923-da50bb15b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:21.344471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6431c-4e4a-4d0c-9923-da50bb15b299 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:21.344745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6431c-4e4a-4d0c-9923-da50bb15b299 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:21.345069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6431c-4e4a-4d0c-9923-da50bb15b299 None admin] PATH_INFO: `/v3/users/22cc2f4064a14ae4985851ddb0c31dec` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:21.346262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6431c-4e4a-4d0c-9923-da50bb15b299 None admin] RBAC: Authorizing `identity:get_user(user_id=22cc2f4064a14ae4985851ddb0c31dec)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:21.347673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6431c-4e4a-4d0c-9923-da50bb15b299 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:21.349247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 583/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:21 2022] GET /identity/v3/users/22cc2f4064a14ae4985851ddb0c31dec => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:21.355109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:21.452884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d02d8-2cf4-4f6c-8cd0-512c53b16a4d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:21.484756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6bfa-a38a-4928-8454-02f19f435a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:21.485333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6bfa-a38a-4928-8454-02f19f435a83 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:21.486165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6bfa-a38a-4928-8454-02f19f435a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:21.486619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c6bfa-a38a-4928-8454-02f19f435a83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:21.530703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:21.552550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c6bfa-a38a-4928-8454-02f19f435a83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:21.553887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c6bfa-a38a-4928-8454-02f19f435a83 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:21.648060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 576/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:17:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:17:21.653947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6431c-4e4a-4d0c-9923-da50bb15b299 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:21.752410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6431c-4e4a-4d0c-9923-da50bb15b299 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:21.784675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01507ada-a28d-425f-b080-51626803e8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:21.785285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507ada-a28d-425f-b080-51626803e8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:21.785560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507ada-a28d-425f-b080-51626803e8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:21.785827 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01507ada-a28d-425f-b080-51626803e8a0 None admin] PATH_INFO: `/v3/projects/030e1cdc9c5747beb3da263345ae463d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:21.786913 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01507ada-a28d-425f-b080-51626803e8a0 None admin] RBAC: Authorizing `identity:get_project(project_id=030e1cdc9c5747beb3da263345ae463d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:21.788343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01507ada-a28d-425f-b080-51626803e8a0 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:21.789938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 584/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:21 2022] GET /identity/v3/projects/030e1cdc9c5747beb3da263345ae463d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:17:21.796119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6bfa-a38a-4928-8454-02f19f435a83 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:21.876392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:21.898355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c6bfa-a38a-4928-8454-02f19f435a83 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:21.932650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b872f7-afdc-4673-9c3b-7ed8b7436ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:21.933241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b872f7-afdc-4673-9c3b-7ed8b7436ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:21.933519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b872f7-afdc-4673-9c3b-7ed8b7436ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:21.933796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b872f7-afdc-4673-9c3b-7ed8b7436ff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:21.934506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b872f7-afdc-4673-9c3b-7ed8b7436ff1 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=030e1cdc9c5747beb3da263345ae463d, user.id=22cc2f4064a14ae4985851ddb0c31dec) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:21.934807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b872f7-afdc-4673-9c3b-7ed8b7436ff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:21.936238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b872f7-afdc-4673-9c3b-7ed8b7436ff1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:21.942255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 577/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:17:21 2022] GET /identity/v3/role_assignments?user.id=22cc2f4064a14ae4985851ddb0c31dec&scope.project.id=030e1cdc9c5747beb3da263345ae463d&role.id=29fd031edd2b4f65aed8942853150f92 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:23.947601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd3b3e-ef2a-4a27-bf43-4c34468952e8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:23.948197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd3b3e-ef2a-4a27-bf43-4c34468952e8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:23.948197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd3b3e-ef2a-4a27-bf43-4c34468952e8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:23.949505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 585/1162] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:23.955276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daaa0d-7b48-40a5-9605-de4f2382e6af None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:23.955666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daaa0d-7b48-40a5-9605-de4f2382e6af None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:23.955917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daaa0d-7b48-40a5-9605-de4f2382e6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:23.974146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-21daaa0d-7b48-40a5-9605-de4f2382e6af None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:23.975326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-21daaa0d-7b48-40a5-9605-de4f2382e6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:24.040748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 578/1163] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:24.062534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd3b3e-ef2a-4a27-bf43-4c34468952e8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:24.112856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:24.163481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd3b3e-ef2a-4a27-bf43-4c34468952e8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:24.198025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e489b7a-d690-4935-a0be-26efc6222ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:24.198558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e489b7a-d690-4935-a0be-26efc6222ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:24.198831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e489b7a-d690-4935-a0be-26efc6222ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:24.199098 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e489b7a-d690-4935-a0be-26efc6222ea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:24.263325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e489b7a-d690-4935-a0be-26efc6222ea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:24.264852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e489b7a-d690-4935-a0be-26efc6222ea9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:24.309891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:24.359928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 586/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:17:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:17:24.366659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21daaa0d-7b48-40a5-9605-de4f2382e6af None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:24.436253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:24.465583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21daaa0d-7b48-40a5-9605-de4f2382e6af None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:24.499159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea5f24-dfd2-43b5-93a7-8b32768b140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:24.499779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea5f24-dfd2-43b5-93a7-8b32768b140c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:24.500056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea5f24-dfd2-43b5-93a7-8b32768b140c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:24.500368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea5f24-dfd2-43b5-93a7-8b32768b140c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:24.506947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea5f24-dfd2-43b5-93a7-8b32768b140c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:24.508405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea5f24-dfd2-43b5-93a7-8b32768b140c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-19ea5f24-dfd2-43b5-93a7-8b32768b140c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:17:24.520852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:17:24.522056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:17:24.523113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 579/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:17:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:17:24.527940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e489b7a-d690-4935-a0be-26efc6222ea9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:24.629747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e489b7a-d690-4935-a0be-26efc6222ea9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:24.662427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344714d-c0e4-49e6-9363-e89901970658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:24.663028 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344714d-c0e4-49e6-9363-e89901970658 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:24.663302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344714d-c0e4-49e6-9363-e89901970658 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:24.663566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344714d-c0e4-49e6-9363-e89901970658 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:24.664218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344714d-c0e4-49e6-9363-e89901970658 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:24.664515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344714d-c0e4-49e6-9363-e89901970658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:24.665862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344714d-c0e4-49e6-9363-e89901970658 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:24.672779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 587/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Aug 11 16:17:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:26.674554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b42a1-ecff-4b4d-8fd7-a30f783475f3 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:26.675119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b42a1-ecff-4b4d-8fd7-a30f783475f3 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:26.675119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b42a1-ecff-4b4d-8fd7-a30f783475f3 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:26.676478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 580/1167] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:26.681638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f565-2337-485e-81f8-57b0ba6a9123 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:26.682029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f565-2337-485e-81f8-57b0ba6a9123 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:26.682206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f565-2337-485e-81f8-57b0ba6a9123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:26.701336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-00a9f565-2337-485e-81f8-57b0ba6a9123 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:26.702532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-00a9f565-2337-485e-81f8-57b0ba6a9123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:26.710234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:26.768834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 588/1168] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:26.791140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b42a1-ecff-4b4d-8fd7-a30f783475f3 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:26.891513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b42a1-ecff-4b4d-8fd7-a30f783475f3 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:26.920675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:26.926067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bd3a6-15c7-4cc4-9d18-9b0e24ed3eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:26.926603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd3a6-15c7-4cc4-9d18-9b0e24ed3eff None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:26.926880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd3a6-15c7-4cc4-9d18-9b0e24ed3eff None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:26.927150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bd3a6-15c7-4cc4-9d18-9b0e24ed3eff None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:26.927817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bd3a6-15c7-4cc4-9d18-9b0e24ed3eff None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:26.929209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bd3a6-15c7-4cc4-9d18-9b0e24ed3eff None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:26.944546 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 581/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Thu Aug 11 16:17:26 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:17:28.957001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e778-788f-40ca-b93b-d3c4e385f4a2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:28.957533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e778-788f-40ca-b93b-d3c4e385f4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:28.957533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617e778-788f-40ca-b93b-d3c4e385f4a2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:28.958885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 589/1170] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:28.964039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a49a9-49f0-42f2-8aee-1f9de093afaa None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:28.964466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a49a9-49f0-42f2-8aee-1f9de093afaa None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:28.964620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a49a9-49f0-42f2-8aee-1f9de093afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:28.982575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-bd3a49a9-49f0-42f2-8aee-1f9de093afaa None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:28.983768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-bd3a49a9-49f0-42f2-8aee-1f9de093afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:29.049339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 582/1171] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:29.073273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617e778-788f-40ca-b93b-d3c4e385f4a2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:29.172851 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617e778-788f-40ca-b93b-d3c4e385f4a2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:29.206909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5aaad3-3f37-473e-86a6-ab85a69eaebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:29.207584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aaad3-3f37-473e-86a6-ab85a69eaebb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:29.207811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aaad3-3f37-473e-86a6-ab85a69eaebb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:29.208081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aaad3-3f37-473e-86a6-ab85a69eaebb None admin] PATH_INFO: `/v3/domains/b0a0b0693ce54228a9f7118e96e35328` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:29.218084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5aaad3-3f37-473e-86a6-ab85a69eaebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0a0b0693ce54228a9f7118e96e35328)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:29.219510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5aaad3-3f37-473e-86a6-ab85a69eaebb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:29.221449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 590/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Thu Aug 11 16:17:29 2022] GET /identity/v3/domains/b0a0b0693ce54228a9f7118e96e35328 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:29.227307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a49a9-49f0-42f2-8aee-1f9de093afaa None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:29.325442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a49a9-49f0-42f2-8aee-1f9de093afaa None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:29.347748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:29.360013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8dd683-bb34-471d-998e-801958b8b49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:29.360739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dd683-bb34-471d-998e-801958b8b49f None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:29.361058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dd683-bb34-471d-998e-801958b8b49f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:29.361324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dd683-bb34-471d-998e-801958b8b49f None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:29.361976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dd683-bb34-471d-998e-801958b8b49f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:29.363392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dd683-bb34-471d-998e-801958b8b49f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:29.389612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 583/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:17:29 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:17:31.428965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98cd5a-df71-4a43-a159-79a1591624b7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:31.429467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98cd5a-df71-4a43-a159-79a1591624b7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:31.429467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98cd5a-df71-4a43-a159-79a1591624b7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:31.430766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 591/1174] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:31.435975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63815-9b64-4597-9722-6ffdda03efc6 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:31.436379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63815-9b64-4597-9722-6ffdda03efc6 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:31.436551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f63815-9b64-4597-9722-6ffdda03efc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:31.455193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-95f63815-9b64-4597-9722-6ffdda03efc6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:31.456454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-95f63815-9b64-4597-9722-6ffdda03efc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:31.521920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 584/1175] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:31.545939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98cd5a-df71-4a43-a159-79a1591624b7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:31.548941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:31.646089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98cd5a-df71-4a43-a159-79a1591624b7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:31.678623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8e291-5028-434a-937e-7a0ff735a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:31.679287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8e291-5028-434a-937e-7a0ff735a6dd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:31.679506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8e291-5028-434a-937e-7a0ff735a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:31.679768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8e291-5028-434a-937e-7a0ff735a6dd None admin] PATH_INFO: `/v3/domains/b0a0b0693ce54228a9f7118e96e35328` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:31.680895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8e291-5028-434a-937e-7a0ff735a6dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=b0a0b0693ce54228a9f7118e96e35328)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:31.682299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8e291-5028-434a-937e-7a0ff735a6dd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:31.683839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 592/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Thu Aug 11 16:17:31 2022] GET /identity/v3/domains/b0a0b0693ce54228a9f7118e96e35328 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:31.690341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63815-9b64-4597-9722-6ffdda03efc6 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:31.786270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f63815-9b64-4597-9722-6ffdda03efc6 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:31.788568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:31.820511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdda849c-a26f-4e77-a70b-34a982dc8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:31.821104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda849c-a26f-4e77-a70b-34a982dc8adc None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:31.821379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda849c-a26f-4e77-a70b-34a982dc8adc None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:31.821642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdda849c-a26f-4e77-a70b-34a982dc8adc None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:31.822280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda849c-a26f-4e77-a70b-34a982dc8adc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:31.823686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdda849c-a26f-4e77-a70b-34a982dc8adc None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:31.845350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 585/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:17:31 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:17:33.887566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bab382-1e46-410c-b0e7-b0a166c90a70 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:33.888188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bab382-1e46-410c-b0e7-b0a166c90a70 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:33.888188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bab382-1e46-410c-b0e7-b0a166c90a70 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:33.889451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 593/1178] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:33.894695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a280c-4f6d-4ede-a885-11b319d7cf13 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:33.895089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a280c-4f6d-4ede-a885-11b319d7cf13 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:33.895245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a280c-4f6d-4ede-a885-11b319d7cf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:33.914197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-957a280c-4f6d-4ede-a885-11b319d7cf13 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:33.915379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-957a280c-4f6d-4ede-a885-11b319d7cf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:33.980243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 586/1179] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:34.004081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bab382-1e46-410c-b0e7-b0a166c90a70 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:34.073066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:34.104981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bab382-1e46-410c-b0e7-b0a166c90a70 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:34.138558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d1ca1-ef53-446e-96fc-9f7e472eb3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:34.139120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d1ca1-ef53-446e-96fc-9f7e472eb3cb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:34.139400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d1ca1-ef53-446e-96fc-9f7e472eb3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:34.139669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d1ca1-ef53-446e-96fc-9f7e472eb3cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:34.149671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d1ca1-ef53-446e-96fc-9f7e472eb3cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-be6d1ca1-ef53-446e-96fc-9f7e472eb3cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:17:34.154333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:17:34.155500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:17:34.156504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 594/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:17:34.161843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a280c-4f6d-4ede-a885-11b319d7cf13 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:34.248074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:34.258839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a280c-4f6d-4ede-a885-11b319d7cf13 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:34.291775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:34.292398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:34.292674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:34.292941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:34.293597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:34.293895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:34.295242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:34.303537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 587/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:34.309245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d1ca1-ef53-446e-96fc-9f7e472eb3cb None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:34.404820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d1ca1-ef53-446e-96fc-9f7e472eb3cb None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:34.432235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:34.438255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc76f7-feba-4804-a887-4580ecd20ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:34.439662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc76f7-feba-4804-a887-4580ecd20ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:34.439936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc76f7-feba-4804-a887-4580ecd20ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:34.440488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc76f7-feba-4804-a887-4580ecd20ed2 None admin] PATH_INFO: `/v3/users/d0548b86003f4dd4b49d31892f8b9694` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:34.453513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc76f7-feba-4804-a887-4580ecd20ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=d0548b86003f4dd4b49d31892f8b9694)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:34.455419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc76f7-feba-4804-a887-4580ecd20ed2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:34.457013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 595/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:34 2022] GET /identity/v3/users/d0548b86003f4dd4b49d31892f8b9694 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:34.462736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:34.559933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf666d22-d9c6-45bc-b459-929a5c79a73d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:34.591946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96bfc8-31a1-4cbe-9473-03091dafd0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:34.592643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96bfc8-31a1-4cbe-9473-03091dafd0ec None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:34.592923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96bfc8-31a1-4cbe-9473-03091dafd0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:34.593194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96bfc8-31a1-4cbe-9473-03091dafd0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:34.614495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:34.659364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96bfc8-31a1-4cbe-9473-03091dafd0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:34.660777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96bfc8-31a1-4cbe-9473-03091dafd0ec None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:34.755670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 588/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:17:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:17:34.761547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc76f7-feba-4804-a887-4580ecd20ed2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:34.856084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc76f7-feba-4804-a887-4580ecd20ed2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:34.888564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e95fac-1549-4b18-8d7b-6ebe3691a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:34.889421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e95fac-1549-4b18-8d7b-6ebe3691a6dd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:34.889698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e95fac-1549-4b18-8d7b-6ebe3691a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:34.889958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e95fac-1549-4b18-8d7b-6ebe3691a6dd None admin] PATH_INFO: `/v3/projects/f2b54792027b4f51a340bd51c720e7be` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:34.891056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e95fac-1549-4b18-8d7b-6ebe3691a6dd None admin] RBAC: Authorizing `identity:get_project(project_id=f2b54792027b4f51a340bd51c720e7be)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:34.893141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e95fac-1549-4b18-8d7b-6ebe3691a6dd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:34.894736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 596/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:34 2022] GET /identity/v3/projects/f2b54792027b4f51a340bd51c720e7be => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:17:34.900819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96bfc8-31a1-4cbe-9473-03091dafd0ec None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:34.951612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:34.997131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96bfc8-31a1-4cbe-9473-03091dafd0ec None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:35.029556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346bbba6-d802-4278-a283-f4ad286dcd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:35.030143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bbba6-d802-4278-a283-f4ad286dcd18 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:35.030416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bbba6-d802-4278-a283-f4ad286dcd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:35.030679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346bbba6-d802-4278-a283-f4ad286dcd18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:35.031392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bbba6-d802-4278-a283-f4ad286dcd18 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=f2b54792027b4f51a340bd51c720e7be, user.id=d0548b86003f4dd4b49d31892f8b9694) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:35.031695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bbba6-d802-4278-a283-f4ad286dcd18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:35.033769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346bbba6-d802-4278-a283-f4ad286dcd18 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:35.038901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 589/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:17:34 2022] GET /identity/v3/role_assignments?user.id=d0548b86003f4dd4b49d31892f8b9694&scope.project.id=f2b54792027b4f51a340bd51c720e7be&role.id=29fd031edd2b4f65aed8942853150f92 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:37.033895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f5ff-acd7-41c3-90a1-64be165a10f7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:37.034463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f5ff-acd7-41c3-90a1-64be165a10f7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:37.034463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4f5ff-acd7-41c3-90a1-64be165a10f7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:37.035762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 597/1186] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:37.041006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441702b-0f26-40f7-8b1a-34f5c381c9dc None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:37.041434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441702b-0f26-40f7-8b1a-34f5c381c9dc None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:37.041541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441702b-0f26-40f7-8b1a-34f5c381c9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:37.059239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c441702b-0f26-40f7-8b1a-34f5c381c9dc None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:37.060482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c441702b-0f26-40f7-8b1a-34f5c381c9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:37.126882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 590/1187] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:37.149236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4f5ff-acd7-41c3-90a1-64be165a10f7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:37.205174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:37.248797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4f5ff-acd7-41c3-90a1-64be165a10f7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:37.280986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a52a22-95e4-4d68-82a8-d7a578f858d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:37.281598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a52a22-95e4-4d68-82a8-d7a578f858d2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:37.281900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a52a22-95e4-4d68-82a8-d7a578f858d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:37.282165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a52a22-95e4-4d68-82a8-d7a578f858d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:37.292435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a52a22-95e4-4d68-82a8-d7a578f858d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-17a52a22-95e4-4d68-82a8-d7a578f858d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:17:37.296994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:17:37.298353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:17:37.299377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 598/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:17:37.304433 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441702b-0f26-40f7-8b1a-34f5c381c9dc None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:37.349046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:37.399211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441702b-0f26-40f7-8b1a-34f5c381c9dc None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:37.431230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:37.431813 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:37.432082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:37.432371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:37.433017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:37.433309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:37.434643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:37.442536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 591/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:37.448389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a52a22-95e4-4d68-82a8-d7a578f858d2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:37.545418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a52a22-95e4-4d68-82a8-d7a578f858d2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:37.561328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:37.580621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7dda2-9752-4d25-b2fa-563ae5b19a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:37.581237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7dda2-9752-4d25-b2fa-563ae5b19a29 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:37.581513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7dda2-9752-4d25-b2fa-563ae5b19a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:37.581782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7dda2-9752-4d25-b2fa-563ae5b19a29 None admin] PATH_INFO: `/v3/users/d0548b86003f4dd4b49d31892f8b9694` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:37.582905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7dda2-9752-4d25-b2fa-563ae5b19a29 None admin] RBAC: Authorizing `identity:get_user(user_id=d0548b86003f4dd4b49d31892f8b9694)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:37.584358 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7dda2-9752-4d25-b2fa-563ae5b19a29 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:37.585917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 599/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:37 2022] GET /identity/v3/users/d0548b86003f4dd4b49d31892f8b9694 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:37.591595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:37.686677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b848f-fa2e-4992-b3f5-37a11f6057ea None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:37.695823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:37.719598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8e331-eed6-43d4-99fb-f9fd2d5c524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:37.720773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8e331-eed6-43d4-99fb-f9fd2d5c524a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:37.721092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8e331-eed6-43d4-99fb-f9fd2d5c524a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:37.721405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8e331-eed6-43d4-99fb-f9fd2d5c524a None admin] PATH_INFO: `/v3/projects/f2b54792027b4f51a340bd51c720e7be` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:37.722549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8e331-eed6-43d4-99fb-f9fd2d5c524a None admin] RBAC: Authorizing `identity:get_project(project_id=f2b54792027b4f51a340bd51c720e7be)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:37.723948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8e331-eed6-43d4-99fb-f9fd2d5c524a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:37.725560 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 592/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:37 2022] GET /identity/v3/projects/f2b54792027b4f51a340bd51c720e7be => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:37.731368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7dda2-9752-4d25-b2fa-563ae5b19a29 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:37.829447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7dda2-9752-4d25-b2fa-563ae5b19a29 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:37.862999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115917c-95cd-4d55-a815-a7d0c66faef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:37.863643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115917c-95cd-4d55-a815-a7d0c66faef1 None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:37.863920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115917c-95cd-4d55-a815-a7d0c66faef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:37.864219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115917c-95cd-4d55-a815-a7d0c66faef1 None admin] PATH_INFO: `/v3/projects/f2b54792027b4f51a340bd51c720e7be/users/d0548b86003f4dd4b49d31892f8b9694/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:37.866087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115917c-95cd-4d55-a815-a7d0c66faef1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2b54792027b4f51a340bd51c720e7be, user_id=d0548b86003f4dd4b49d31892f8b9694, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:37.867511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115917c-95cd-4d55-a815-a7d0c66faef1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:37.880157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 600/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:17:37 2022] PUT /identity/v3/projects/f2b54792027b4f51a340bd51c720e7be/users/d0548b86003f4dd4b49d31892f8b9694/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:17:39.851719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023f71-dccf-4649-94ce-e338d1a39b8a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:39.852311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023f71-dccf-4649-94ce-e338d1a39b8a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:39.852311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49023f71-dccf-4649-94ce-e338d1a39b8a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:39.853643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 593/1193] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:17:39.858650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1df0c-7e17-44b0-b412-36e084bd9d78 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:39.859039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1df0c-7e17-44b0-b412-36e084bd9d78 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:39.859187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1df0c-7e17-44b0-b412-36e084bd9d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:39.877335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-87e1df0c-7e17-44b0-b412-36e084bd9d78 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:17:39.878508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-87e1df0c-7e17-44b0-b412-36e084bd9d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:17:39.917808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:39.945120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 601/1194] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:17:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:17:39.968429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49023f71-dccf-4649-94ce-e338d1a39b8a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:40.068142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49023f71-dccf-4649-94ce-e338d1a39b8a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:40.100977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acba4f-2d60-4370-8153-9030ec755172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:40.101579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acba4f-2d60-4370-8153-9030ec755172 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:40.101914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acba4f-2d60-4370-8153-9030ec755172 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:40.102131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acba4f-2d60-4370-8153-9030ec755172 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:40.112395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acba4f-2d60-4370-8153-9030ec755172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-d8acba4f-2d60-4370-8153-9030ec755172 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:17:40.117006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:17:40.118101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:17:40.119173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 594/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:17:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:17:40.124124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1df0c-7e17-44b0-b412-36e084bd9d78 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:40.224953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1df0c-7e17-44b0-b412-36e084bd9d78 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:40.257581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:40.258174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:40.258446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:40.258710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:40.259352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:40.259643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:40.261002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:40.268761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 602/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:17:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:40.274485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acba4f-2d60-4370-8153-9030ec755172 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:40.308449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:40.372049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acba4f-2d60-4370-8153-9030ec755172 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:40.405168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5ba94-01ea-4ca8-abb0-358e4d51ccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:40.405786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5ba94-01ea-4ca8-abb0-358e4d51ccdb None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:40.406067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5ba94-01ea-4ca8-abb0-358e4d51ccdb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:40.406339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5ba94-01ea-4ca8-abb0-358e4d51ccdb None admin] PATH_INFO: `/v3/users/d0548b86003f4dd4b49d31892f8b9694` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:40.407652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5ba94-01ea-4ca8-abb0-358e4d51ccdb None admin] RBAC: Authorizing `identity:get_user(user_id=d0548b86003f4dd4b49d31892f8b9694)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:40.409098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5ba94-01ea-4ca8-abb0-358e4d51ccdb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:40.410665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 595/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Aug 11 16:17:40 2022] GET /identity/v3/users/d0548b86003f4dd4b49d31892f8b9694 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:40.416635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:40.451475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:40.513900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d380cf-650b-47fa-ad02-396d0727f733 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:40.545799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b69a-b0b6-4e35-b140-bb8be58d1979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:40.546388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b69a-b0b6-4e35-b140-bb8be58d1979 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:40.546661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b69a-b0b6-4e35-b140-bb8be58d1979 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:40.546927 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b69a-b0b6-4e35-b140-bb8be58d1979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:40.611333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163b69a-b0b6-4e35-b140-bb8be58d1979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:40.612814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163b69a-b0b6-4e35-b140-bb8be58d1979 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:40.644546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:40.707087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 603/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:17:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:17:40.713385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5ba94-01ea-4ca8-abb0-358e4d51ccdb None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:40.809597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5ba94-01ea-4ca8-abb0-358e4d51ccdb None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:40.811824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:40.843064 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523b9a1-13ba-4d77-82af-b12df8486462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:40.843675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523b9a1-13ba-4d77-82af-b12df8486462 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:40.843948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523b9a1-13ba-4d77-82af-b12df8486462 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:40.844243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523b9a1-13ba-4d77-82af-b12df8486462 None admin] PATH_INFO: `/v3/projects/f2b54792027b4f51a340bd51c720e7be` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:40.845866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523b9a1-13ba-4d77-82af-b12df8486462 None admin] RBAC: Authorizing `identity:get_project(project_id=f2b54792027b4f51a340bd51c720e7be)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:40.847286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523b9a1-13ba-4d77-82af-b12df8486462 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:40.848918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 596/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Aug 11 16:17:40 2022] GET /identity/v3/projects/f2b54792027b4f51a340bd51c720e7be => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:40.855126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b69a-b0b6-4e35-b140-bb8be58d1979 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:17:40.949784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b69a-b0b6-4e35-b140-bb8be58d1979 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:17:40.982543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54bb21-f777-44b0-9f48-328a41d25daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:17:40.983129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54bb21-f777-44b0-9f48-328a41d25daf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:17:40.983403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54bb21-f777-44b0-9f48-328a41d25daf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:17:40.983667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54bb21-f777-44b0-9f48-328a41d25daf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:17:40.984410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54bb21-f777-44b0-9f48-328a41d25daf None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=f2b54792027b4f51a340bd51c720e7be, user.id=d0548b86003f4dd4b49d31892f8b9694) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:17:40.984713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54bb21-f777-44b0-9f48-328a41d25daf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:17:40.986491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54bb21-f777-44b0-9f48-328a41d25daf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:17:40.989277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:17:40.995482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 604/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:17:40 2022] GET /identity/v3/role_assignments?user.id=d0548b86003f4dd4b49d31892f8b9694&scope.project.id=f2b54792027b4f51a340bd51c720e7be&role.id=29fd031edd2b4f65aed8942853150f92 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:17:50.839025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=14 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:36.981618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6031056-cdd1-48c3-8e56-7a2b29739987 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:36.982470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6031056-cdd1-48c3-8e56-7a2b29739987 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:36.982470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6031056-cdd1-48c3-8e56-7a2b29739987 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:36.984137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 597/1201] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:18:36.991547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93019399-82f3-4413-9099-536c053b5e46 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:36.992158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93019399-82f3-4413-9099-536c053b5e46 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:36.992354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93019399-82f3-4413-9099-536c053b5e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:37.013470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-93019399-82f3-4413-9099-536c053b5e46 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:18:37.014918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-93019399-82f3-4413-9099-536c053b5e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:18:37.110293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 605/1202] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:18:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:18:37.139953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6031056-cdd1-48c3-8e56-7a2b29739987 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:37.271585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6031056-cdd1-48c3-8e56-7a2b29739987 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:37.316498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:37.317391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:37.317742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:37.317957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:37.328933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:37.331101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:37.353540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:37.354942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:37.356206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:37.356206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:37.356206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:37.359605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 598/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:37.361404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93019399-82f3-4413-9099-536c053b5e46 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:37.524273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93019399-82f3-4413-9099-536c053b5e46 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:37.560482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:37.568215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac1146-c217-40ae-971a-0edb3187842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:37.568954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1146-c217-40ae-971a-0edb3187842c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:37.569228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1146-c217-40ae-971a-0edb3187842c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:37.569529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac1146-c217-40ae-971a-0edb3187842c None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:37.570236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac1146-c217-40ae-971a-0edb3187842c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:37.570535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac1146-c217-40ae-971a-0edb3187842c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:37.572226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac1146-c217-40ae-971a-0edb3187842c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:37.589906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 606/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:18:37.597216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:37.668658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:37.726290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a64101-30fc-47d1-b89b-6ea787a601cb None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:37.769870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d169f7-33ef-40a3-95cd-920913ca4aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:37.770497 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d169f7-33ef-40a3-95cd-920913ca4aa9 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:37.770775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d169f7-33ef-40a3-95cd-920913ca4aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:37.771045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d169f7-33ef-40a3-95cd-920913ca4aa9 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:37.771705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d169f7-33ef-40a3-95cd-920913ca4aa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:37.773175 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d169f7-33ef-40a3-95cd-920913ca4aa9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:37.808223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 599/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Aug 11 16:18:37 2022] POST /identity/v3/users => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:18:40.020761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591ce83-b2e1-4abe-9ff9-9001563bebe8 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:40.021358 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591ce83-b2e1-4abe-9ff9-9001563bebe8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:40.021358 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591ce83-b2e1-4abe-9ff9-9001563bebe8 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:40.022681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 607/1206] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:18:40.028732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a469-230b-47ec-b318-c329e7d9b17c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:40.029229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a469-230b-47ec-b318-c329e7d9b17c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:40.029380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932a469-230b-47ec-b318-c329e7d9b17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:40.049514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3932a469-230b-47ec-b318-c329e7d9b17c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:18:40.053848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3932a469-230b-47ec-b318-c329e7d9b17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:18:40.132704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 600/1207] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:18:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:18:40.157466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591ce83-b2e1-4abe-9ff9-9001563bebe8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:40.275121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591ce83-b2e1-4abe-9ff9-9001563bebe8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:40.315568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31770938-255b-4a27-aec1-0cd2ac00c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:40.316812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31770938-255b-4a27-aec1-0cd2ac00c6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:40.317100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31770938-255b-4a27-aec1-0cd2ac00c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:40.317719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31770938-255b-4a27-aec1-0cd2ac00c6f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:40.330796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31770938-255b-4a27-aec1-0cd2ac00c6f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-31770938-255b-4a27-aec1-0cd2ac00c6f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:18:40.335914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:40.336937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:18:40.337908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:40.342285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 608/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:18:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:40.343182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932a469-230b-47ec-b318-c329e7d9b17c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:40.388774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:40.464658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932a469-230b-47ec-b318-c329e7d9b17c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:40.503803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:40.504440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:40.504714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:40.504954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:40.505641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:40.506049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:40.507314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:40.518057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 601/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:40.524337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31770938-255b-4a27-aec1-0cd2ac00c6f7 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:40.582682 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:40.636663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31770938-255b-4a27-aec1-0cd2ac00c6f7 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:40.674261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:40.675046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:40.675556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:40.675928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:40.686602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:40.688202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:40.704123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:40.706135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:40.707251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:40.707251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:40.707251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:40.707251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 609/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:40.713751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:40.828678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:40.836193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703c5b3-dd31-4515-bdd7-0b24d67d6a2b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:40.874494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:40.875196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:40.876631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:40.876631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:40.877111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:40.877414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:40.878966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:40.894498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 602/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:40.899962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:41.035365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14588ed-a816-4d80-a1b2-4a05c22ac84a None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:41.038328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:41.081594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:41.082298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:41.082755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:41.083223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:41.092209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:41.093606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:18:41.107627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:41.108968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:41.109966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 610/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:18:41.115476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:41.241108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c57d0-a9ad-4151-af7a-1b912f53f5e4 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:41.285317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:41.286137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:41.286428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:41.286737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:41.287777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:41.288131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:41.289659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:41.312056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 603/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Aug 11 16:18:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:41.318414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:41.452407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97717a-8fbe-4492-b747-8aa67351876e None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:41.495754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee7699-1af1-4b3c-8353-842ddf589e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:41.496967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7699-1af1-4b3c-8353-842ddf589e42 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:41.496967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7699-1af1-4b3c-8353-842ddf589e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:41.497182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee7699-1af1-4b3c-8353-842ddf589e42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:41.530045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:41.575948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee7699-1af1-4b3c-8353-842ddf589e42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:41.577697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee7699-1af1-4b3c-8353-842ddf589e42 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:41.693008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 611/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:18:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 11 16:18:41.698587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:41.722762 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:41.816734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b649708-538c-41e3-8580-c75b52468783 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:41.856495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:41.857265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:41.858170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:41.858170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:41.869972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:41.871427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:41.889184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:41.890334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:41.891252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:41.891252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:41.891252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:41.891252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 604/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:18:41.897254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee7699-1af1-4b3c-8353-842ddf589e42 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:41.965263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:42.013283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee7699-1af1-4b3c-8353-842ddf589e42 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:42.056318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:42.056783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:42.056783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:42.057044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:42.057874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:42.058948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:42.059679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:42.076129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 612/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:42.081290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:42.171976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:42.207977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c79d6-664f-4e87-9e85-7bafa05bb28f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:42.248682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:42.249332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:42.249608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:42.249877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:42.259147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:42.260689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:18:42.273106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:42.274187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:42.275109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 605/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:42.280462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:42.399349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca64ef-6d3c-44a4-b29c-9202a6ec6a33 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:42.426426 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:42.442077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e0ed5-c66d-4b9d-9007-5e3ea0702cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:42.443098 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e0ed5-c66d-4b9d-9007-5e3ea0702cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:42.443479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e0ed5-c66d-4b9d-9007-5e3ea0702cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:42.443851 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e0ed5-c66d-4b9d-9007-5e3ea0702cf7 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:42.444674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e0ed5-c66d-4b9d-9007-5e3ea0702cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:42.445078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e0ed5-c66d-4b9d-9007-5e3ea0702cf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:42.446653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e0ed5-c66d-4b9d-9007-5e3ea0702cf7 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:42.463494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 613/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:42.469467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:42.577891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54289a3e-8887-4ba5-ae3d-32b3da45be04 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:42.611954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:42.621789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26214a-09c8-4107-9517-9b2212793f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:42.622531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26214a-09c8-4107-9517-9b2212793f7c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:42.622812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26214a-09c8-4107-9517-9b2212793f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:42.623127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26214a-09c8-4107-9517-9b2212793f7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:42.624017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26214a-09c8-4107-9517-9b2212793f7c None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=08abc209f6cd4b62b34cad53c8ba4c54) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:42.624534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26214a-09c8-4107-9517-9b2212793f7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:42.626868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26214a-09c8-4107-9517-9b2212793f7c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:42.636172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 606/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:18:42 2022] GET /identity/v3/role_assignments?user.id=08abc209f6cd4b62b34cad53c8ba4c54&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:44.872777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543ef8-7d18-4523-a2cf-4863bc449c2c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:44.873337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543ef8-7d18-4523-a2cf-4863bc449c2c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:44.873337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543ef8-7d18-4523-a2cf-4863bc449c2c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:44.874648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 614/1220] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:18:44.880431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f294758-647e-40d5-9231-b625ccf768d5 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:44.880886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f294758-647e-40d5-9231-b625ccf768d5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:44.880990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f294758-647e-40d5-9231-b625ccf768d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:44.902038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-7f294758-647e-40d5-9231-b625ccf768d5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:18:44.903381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7f294758-647e-40d5-9231-b625ccf768d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:18:44.980127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 607/1221] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:18:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:18:45.000540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19543ef8-7d18-4523-a2cf-4863bc449c2c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:45.116769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19543ef8-7d18-4523-a2cf-4863bc449c2c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:45.152207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd079b1-4b6d-40ba-b69e-a976d774f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:45.153011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd079b1-4b6d-40ba-b69e-a976d774f03c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:45.153337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd079b1-4b6d-40ba-b69e-a976d774f03c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:45.153609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd079b1-4b6d-40ba-b69e-a976d774f03c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:45.165337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd079b1-4b6d-40ba-b69e-a976d774f03c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-6dd079b1-4b6d-40ba-b69e-a976d774f03c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:18:45.170635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:45.171698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:45.172594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 615/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:18:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:45.178951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f294758-647e-40d5-9231-b625ccf768d5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:45.285039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f294758-647e-40d5-9231-b625ccf768d5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:45.310149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:45.322926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:45.323528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:45.323801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:45.324068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:45.324749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:45.325044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:45.326468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:45.335321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 608/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:18:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:18:45.341401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd079b1-4b6d-40ba-b69e-a976d774f03c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:45.390108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:45.458041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd079b1-4b6d-40ba-b69e-a976d774f03c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:45.497906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:45.498541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:45.498820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:45.499091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:45.510354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:45.511804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:45.526000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:45.527086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:45.527998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:45.527998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:45.527998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:45.527998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 616/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:45.533441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:45.640561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b8df-a8b4-44ba-8010-d360ad9619e3 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:45.681912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:45.682511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:45.682784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:45.683048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:45.683689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:45.684010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:45.685397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:45.698450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 609/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:45.704314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:45.803102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:45.810339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e7e33-778a-4c45-91ff-0e6d0b5741c5 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:45.848870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:45.849576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:45.849899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:45.850206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:45.858735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:45.861264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:18:45.874562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:45.876178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:45.877453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 617/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:45.882334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:45.933927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:45.991770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796102cf-7e7f-42ce-ab42-e46de0d01549 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:46.031192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:46.031852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:46.032159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:46.033295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:46.034132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:46.034462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:46.036209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:46.056077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 610/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Aug 11 16:18:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:18:46.061701 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:46.170594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a6fd2-0a68-4eff-8850-5303e7da9e72 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:46.209906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:46.210666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:46.211022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:46.211363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:46.224553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:46.226155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:46.243349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:46.244586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:46.245515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:46.245515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:46.245515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:46.246566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 618/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:46.252899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:46.368966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978976d-f811-4ca8-9006-52a158f1f456 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:46.371527 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:46.407883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:46.408533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:46.408799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:46.409093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:46.409861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:46.410173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:46.411967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:46.425334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 611/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:18:46.433655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:46.444593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:46.544312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc0dd9-0545-4af8-bb30-9a2b7a91f75c None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:46.579164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:46.579791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:46.580066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:46.580362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:46.586860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:46.588393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:18:46.600800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:46.601931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:46.603327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:18:46.603327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:18:46.603327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:18:46.603327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:18:46.603327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:18:46.603327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:18:46.603327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:46.603327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:46.604333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 619/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:46.608382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:46.729352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5677a-da10-4640-8290-54b40b74a984 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:46.770062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b955bc-e6cb-454d-9979-625807a16187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:46.770683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b955bc-e6cb-454d-9979-625807a16187 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:46.770935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b955bc-e6cb-454d-9979-625807a16187 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:46.771195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b955bc-e6cb-454d-9979-625807a16187 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:46.771893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b955bc-e6cb-454d-9979-625807a16187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:46.772444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b955bc-e6cb-454d-9979-625807a16187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:46.773941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b955bc-e6cb-454d-9979-625807a16187 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:46.785880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 612/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:46.792294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:46.870626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:46.911272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65a545-0117-4474-a28b-0c1d0c5b6a7f None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:46.947723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48d02a-858e-4b9b-8135-3624b5e63c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:46.948410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48d02a-858e-4b9b-8135-3624b5e63c8f None admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:46.948684 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48d02a-858e-4b9b-8135-3624b5e63c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:46.948953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48d02a-858e-4b9b-8135-3624b5e63c8f None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/08abc209f6cd4b62b34cad53c8ba4c54/roles/7bd0567f3a3c43628a4ec259a52f1e24` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:46.963453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48d02a-858e-4b9b-8135-3624b5e63c8f None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=08abc209f6cd4b62b34cad53c8ba4c54, role_id=7bd0567f3a3c43628a4ec259a52f1e24)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:46.964916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48d02a-858e-4b9b-8135-3624b5e63c8f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:46.978365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 620/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:18:46 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/08abc209f6cd4b62b34cad53c8ba4c54/roles/7bd0567f3a3c43628a4ec259a52f1e24 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:18:49.091680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9df6-fc54-4414-956d-aa894fec8453 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:49.092411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9df6-fc54-4414-956d-aa894fec8453 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:49.092411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9df6-fc54-4414-956d-aa894fec8453 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:49.093838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 613/1233] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:18:49.099845 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40f0db-08fc-4dd1-b3e2-969d4903765e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:49.100445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40f0db-08fc-4dd1-b3e2-969d4903765e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:49.100547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40f0db-08fc-4dd1-b3e2-969d4903765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:49.120281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0a40f0db-08fc-4dd1-b3e2-969d4903765e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:18:49.121772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0a40f0db-08fc-4dd1-b3e2-969d4903765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:18:49.200698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 621/1234] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:18:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:18:49.225887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9df6-fc54-4414-956d-aa894fec8453 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:49.253777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:49.347868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9df6-fc54-4414-956d-aa894fec8453 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:49.389946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ecccd-4760-4f78-b634-32dd878c4a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:49.390659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ecccd-4760-4f78-b634-32dd878c4a0b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:49.390852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ecccd-4760-4f78-b634-32dd878c4a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:49.391124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ecccd-4760-4f78-b634-32dd878c4a0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:49.401670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ecccd-4760-4f78-b634-32dd878c4a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-ad9ecccd-4760-4f78-b634-32dd878c4a0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:18:49.406307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:49.407324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:49.408528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 614/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Aug 11 16:18:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:49.413810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40f0db-08fc-4dd1-b3e2-969d4903765e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:49.461716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:49.520309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40f0db-08fc-4dd1-b3e2-969d4903765e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:49.557863 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:49.558485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:49.558765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:49.559034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:49.560220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] RBAC: Adding query filter params (name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:49.560710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:49.562615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:49.574368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 622/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Aug 11 16:18:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:49.580393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ecccd-4760-4f78-b634-32dd878c4a0b None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:49.686479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:49.705843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ecccd-4760-4f78-b634-32dd878c4a0b None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:49.745274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:49.745910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:49.746189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:49.746461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:49.757448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:49.758882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:49.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:49.775455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:49.776929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:49.776929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:49.776929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:49.776929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 615/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:49.781212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:49.896014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89040c-ba62-4d58-baeb-8ddc391b7e24 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:49.907225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:49.936114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:49.936810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:49.937188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:49.937537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:49.939150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:49.939512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:49.941061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:49.959453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 623/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:49.960699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:50.073314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8148b-6ab6-4b39-9dbe-fb0abd984600 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:50.110093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:50.111460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:50.111622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:50.111947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:50.115450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:50.121457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:50.123075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:18:50.136595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:50.137659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:50.138624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 616/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:50.144297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:50.256059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bac0e-9d94-4f11-851f-04c440a3df09 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:50.296158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:50.296784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:50.297073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:50.297380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:50.298744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:50.299069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:50.300528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:50.318185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 624/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Aug 11 16:18:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:50.324413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:50.434623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4c130-5135-403b-adb4-90de270ce1e8 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:50.474893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42457e-cd88-477c-b6fe-1ad221bc2fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:50.475596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42457e-cd88-477c-b6fe-1ad221bc2fcd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:50.475858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42457e-cd88-477c-b6fe-1ad221bc2fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:50.476230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42457e-cd88-477c-b6fe-1ad221bc2fcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:50.548661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:50.556332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42457e-cd88-477c-b6fe-1ad221bc2fcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:50.558000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42457e-cd88-477c-b6fe-1ad221bc2fcd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:50.677052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 617/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:18:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:18:50.684021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:50.716261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:50.797349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a74823-75c3-4839-ac05-0d88a59b31ba None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:50.840989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:50.841651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:50.842229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:50.842526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:50.857056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:50.858508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:50.882317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:50.883806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:50.885163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:50.885163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:50.885163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:50.885163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 625/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:50.889971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42457e-cd88-477c-b6fe-1ad221bc2fcd None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:50.992399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:50.999535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42457e-cd88-477c-b6fe-1ad221bc2fcd None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:51.039146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:51.039751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:51.040028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:51.040346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:51.040990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:51.041289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:51.042973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:51.056129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 618/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:51.061316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:51.149148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:51.171649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8494d7e-abdc-41d1-b963-688775de8a07 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:51.211552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:51.212355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:51.212704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:51.212975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:51.221585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:51.223051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:18:51.236568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:51.237778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:51.239044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 626/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:18:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:51.243893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:51.364990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6004bf8-0a9d-4940-82ff-f048fac136d6 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:51.405972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41eda0-046a-4847-a319-832a5c37c14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:51.406704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41eda0-046a-4847-a319-832a5c37c14b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:51.407017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41eda0-046a-4847-a319-832a5c37c14b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:51.407979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41eda0-046a-4847-a319-832a5c37c14b None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:51.408994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41eda0-046a-4847-a319-832a5c37c14b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:51.409354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41eda0-046a-4847-a319-832a5c37c14b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:51.411448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41eda0-046a-4847-a319-832a5c37c14b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:51.426088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 619/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:18:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:51.432537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:51.540700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44705b4f-270b-41c1-866c-1364cbbaf49d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:51.581154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac5103-0ec2-480e-b30c-bcb1868bcd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:51.581816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5103-0ec2-480e-b30c-bcb1868bcd71 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:51.582115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5103-0ec2-480e-b30c-bcb1868bcd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:51.582460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5103-0ec2-480e-b30c-bcb1868bcd71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:51.583236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac5103-0ec2-480e-b30c-bcb1868bcd71 None admin] RBAC: Adding query filter params (role.id=7bd0567f3a3c43628a4ec259a52f1e24, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=08abc209f6cd4b62b34cad53c8ba4c54) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:51.583539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac5103-0ec2-480e-b30c-bcb1868bcd71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:51.585003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac5103-0ec2-480e-b30c-bcb1868bcd71 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:51.591394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 627/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:18:51 2022] GET /identity/v3/role_assignments?user.id=08abc209f6cd4b62b34cad53c8ba4c54&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=7bd0567f3a3c43628a4ec259a52f1e24 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:53.906588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29621dfe-5380-48bd-8803-2b7b4bf81bb6 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:53.907216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29621dfe-5380-48bd-8803-2b7b4bf81bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:53.907216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29621dfe-5380-48bd-8803-2b7b4bf81bb6 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:53.908758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 620/1247] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:18:53.914543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4afd4-9546-40e3-b2b5-57b7c3039085 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:53.915040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4afd4-9546-40e3-b2b5-57b7c3039085 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:53.915158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4afd4-9546-40e3-b2b5-57b7c3039085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:53.921672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:53.934642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d6d4afd4-9546-40e3-b2b5-57b7c3039085 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:18:53.935852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d6d4afd4-9546-40e3-b2b5-57b7c3039085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:18:54.014215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 628/1248] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:18:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:18:54.040909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29621dfe-5380-48bd-8803-2b7b4bf81bb6 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:54.061049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:54.158007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29621dfe-5380-48bd-8803-2b7b4bf81bb6 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:54.197466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8de6c-4b31-48e2-be20-bd904ac494ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:54.198131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8de6c-4b31-48e2-be20-bd904ac494ff None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:54.198398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8de6c-4b31-48e2-be20-bd904ac494ff None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:54.198663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8de6c-4b31-48e2-be20-bd904ac494ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:54.210730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8de6c-4b31-48e2-be20-bd904ac494ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-38a8de6c-4b31-48e2-be20-bd904ac494ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:18:54.215360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:54.216585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:54.217641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 621/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:54.224131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4afd4-9546-40e3-b2b5-57b7c3039085 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:54.335142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4afd4-9546-40e3-b2b5-57b7c3039085 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:54.370850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:54.377695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:54.378374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:54.378645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:54.378919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:54.379689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:54.380027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:54.381551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:54.390639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 629/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:18:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:54.397738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8de6c-4b31-48e2-be20-bd904ac494ff None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:54.486125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:54.515971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8de6c-4b31-48e2-be20-bd904ac494ff None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:54.554976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:54.555611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:54.555874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:54.556170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:54.567848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:54.569335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:54.583883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:54.585135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:54.586423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:54.586423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:54.586423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:54.586423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 622/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:54.591449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:54.721178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb16865-8b9d-43b9-8b45-c3fb06f574b1 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:54.761017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:54.761551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:54.761830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:54.762098 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:54.764768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:54.765077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:54.768380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:54.785397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 630/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:18:54.794962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:54.916663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082c88-bfe6-4c16-9c21-9f6eeb496095 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:54.945908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:54.954954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:54.955582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:54.955869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:54.956170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:54.964695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:54.966057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:18:54.979824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:54.981027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:54.982018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 623/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 11 16:18:54.987518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:55.053508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:55.112471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914f71ad-0215-4b3e-91a4-fe67704b4978 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:55.154975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:55.155584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:55.155911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:55.156280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:55.157139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:55.157730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:55.159430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:55.181194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 631/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Aug 11 16:18:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:55.187589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:55.295236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82f9b6-a70d-4d25-9240-86c73b677413 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:55.334421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517aa262-31ef-4025-8330-a1cde9e91792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:55.334939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517aa262-31ef-4025-8330-a1cde9e91792 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:55.335212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517aa262-31ef-4025-8330-a1cde9e91792 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:55.335479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517aa262-31ef-4025-8330-a1cde9e91792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:55.394718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:55.407039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517aa262-31ef-4025-8330-a1cde9e91792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:55.408706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517aa262-31ef-4025-8330-a1cde9e91792 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:55.525472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 624/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:18:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:18:55.531707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:55.646173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5c948-a249-4786-a5ad-1354a1c68658 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:55.660360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:55.691238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:55.692021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:55.692328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:55.692946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:55.706048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:55.707366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:55.723094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:55.724300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:55.725222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:55.725222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:55.725222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:55.725222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 632/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:55.730283 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517aa262-31ef-4025-8330-a1cde9e91792 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:55.821924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:55.837310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517aa262-31ef-4025-8330-a1cde9e91792 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:55.877774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:55.878401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:55.878682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:55.878952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:55.879599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:55.879900 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:55.881376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:55.893610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 625/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:55.899480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:56.013941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ed4c7-9bfc-4622-af54-72ee0ba7f0a9 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:56.052876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:56.053511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:56.053789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:56.054059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:56.061779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:56.063229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:18:56.077609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:56.078743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:18:56.079790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 633/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:56.085045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:56.199293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ccbea6-24d5-4ed2-8b51-cf6beba1dd9d None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:56.234534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409118bb-dc28-4d18-b7be-ee14e714c526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:56.235156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409118bb-dc28-4d18-b7be-ee14e714c526 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:56.235971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409118bb-dc28-4d18-b7be-ee14e714c526 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:56.236273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409118bb-dc28-4d18-b7be-ee14e714c526 None admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:56.237021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409118bb-dc28-4d18-b7be-ee14e714c526 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:56.237346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409118bb-dc28-4d18-b7be-ee14e714c526 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:56.238835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409118bb-dc28-4d18-b7be-ee14e714c526 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:56.252509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 626/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:56.258627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:56.289943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:56.370391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520bb6a-4cb1-44af-894c-adebf122175d None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:56.405304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e762f-2006-4bf1-8553-21799dcdcde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:56.405923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e762f-2006-4bf1-8553-21799dcdcde9 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:56.406456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e762f-2006-4bf1-8553-21799dcdcde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:56.406456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e762f-2006-4bf1-8553-21799dcdcde9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:56.407722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e762f-2006-4bf1-8553-21799dcdcde9 None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=08abc209f6cd4b62b34cad53c8ba4c54) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:56.408255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e762f-2006-4bf1-8553-21799dcdcde9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:56.409698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e762f-2006-4bf1-8553-21799dcdcde9 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:56.416033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 634/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:18:56 2022] GET /identity/v3/role_assignments?user.id=08abc209f6cd4b62b34cad53c8ba4c54&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=29fd031edd2b4f65aed8942853150f92 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:58.420040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbef2f-dd90-4edf-97f3-a5ed5d27e2be None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:58.420575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbef2f-dd90-4edf-97f3-a5ed5d27e2be None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:58.420575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbef2f-dd90-4edf-97f3-a5ed5d27e2be None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:58.421851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 627/1261] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:18:58.427601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7533a-c030-4f7d-9941-9496751905b3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:58.427981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7533a-c030-4f7d-9941-9496751905b3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:58.428167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7533a-c030-4f7d-9941-9496751905b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:58.446421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d9c7533a-c030-4f7d-9941-9496751905b3 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:18:58.447584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d9c7533a-c030-4f7d-9941-9496751905b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:18:58.512158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 635/1262] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:18:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 11 16:18:58.534512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbef2f-dd90-4edf-97f3-a5ed5d27e2be None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:58.545373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:58.633900 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbef2f-dd90-4edf-97f3-a5ed5d27e2be None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:58.676494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b625fae-d1cc-412c-ab9e-4ef2f0c87dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:58.677198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b625fae-d1cc-412c-ab9e-4ef2f0c87dde None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:58.677481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b625fae-d1cc-412c-ab9e-4ef2f0c87dde None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:58.677785 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b625fae-d1cc-412c-ab9e-4ef2f0c87dde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:58.690881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b625fae-d1cc-412c-ab9e-4ef2f0c87dde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-6b625fae-d1cc-412c-ab9e-4ef2f0c87dde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:18:58.696440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:58.697932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:58.699400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 628/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:18:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:58.704813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7533a-c030-4f7d-9941-9496751905b3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:58.724055 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:58.801662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7533a-c030-4f7d-9941-9496751905b3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:58.834096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:58.834686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:58.834963 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:58.835233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:58.835891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:58.836222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:58.837577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:58.845816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 636/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:18:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:58.851698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b625fae-d1cc-412c-ab9e-4ef2f0c87dde None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:58.917723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:58.951026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b625fae-d1cc-412c-ab9e-4ef2f0c87dde None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:58.984349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:58.984992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:58.985271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:58.985616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:58.995278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:58.996845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:59.010781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:59.011861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:59.013039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:59.013039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:59.013039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:59.013039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 629/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:59.018411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:59.099188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:59.117724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15f183-7f9e-4dc9-82ce-5aa096411bcf None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:59.150184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:59.150785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:59.151064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:59.151336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:59.151976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:59.152307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:59.153665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:59.165483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 637/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:18:59.171100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:59.268846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1c863-d645-4e34-ab1c-c59e22dd54d5 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:59.287439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:59.301747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:59.302366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:59.302647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:59.302918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:59.311051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:59.312533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:18:59.324055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:59.330435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:59.331686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 630/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:59.332435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:59.428317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b201295-e85f-457f-b348-2cf5548b67e2 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:59.461754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:59.462341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:59.462618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:59.462887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:59.463605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:59.463907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:59.465349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:59.473919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:59.482852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 638/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Aug 11 16:18:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:59.488594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:59.585589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e806e3-4837-48ab-beb5-71414b3c32c1 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:59.617711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:59.618318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:59.618594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:59.618860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:59.629368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:59.630794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:18:59.646179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:59.647271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:18:59.648252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:18:59.648252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:18:59.648252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:59.648252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 631/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:59.653951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:59.762769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39dab4-949c-434b-8d9d-c89d3e5a84a3 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:59.795235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:59.795832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:59.796129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:59.796402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:59.797032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:18:59.797329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:59.799336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:59.809960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 639/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:18:59.816917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:18:59.847514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:18:59.913887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1b79a-28cd-41e3-9f29-3c3cc9e978f9 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:18:59.946711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:18:59.947310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:18:59.947590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:18:59.947856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:18:59.954146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:18:59.955618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:18:59.966472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:18:59.967537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:18:59.968860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 632/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:18:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:18:59.973409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:00.029715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:00.072451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed244f-f1b9-404c-a8f8-8faf390e6e19 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:00.104983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22d679-9073-43a7-9e47-6bb43b57936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:00.105522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22d679-9073-43a7-9e47-6bb43b57936a None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:00.105801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22d679-9073-43a7-9e47-6bb43b57936a None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:00.106073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22d679-9073-43a7-9e47-6bb43b57936a None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:00.106782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22d679-9073-43a7-9e47-6bb43b57936a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:00.107087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22d679-9073-43a7-9e47-6bb43b57936a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:00.108532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22d679-9073-43a7-9e47-6bb43b57936a None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:00.120569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 640/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:00.126519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:00.219337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:00.229096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5ac55-a719-4c5d-9b10-bb5aa90d532a None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:00.261835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75491857-1543-4149-914c-6d8df998ad74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:00.262470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75491857-1543-4149-914c-6d8df998ad74 None admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:00.262744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75491857-1543-4149-914c-6d8df998ad74 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:00.263010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75491857-1543-4149-914c-6d8df998ad74 None admin] PATH_INFO: `/v3/projects/2d968a7efd10472db2d392353d807180/users/08abc209f6cd4b62b34cad53c8ba4c54/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:00.264878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75491857-1543-4149-914c-6d8df998ad74 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d968a7efd10472db2d392353d807180, user_id=08abc209f6cd4b62b34cad53c8ba4c54, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:00.266283 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75491857-1543-4149-914c-6d8df998ad74 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:00.278253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 633/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Aug 11 16:19:00 2022] PUT /identity/v3/projects/2d968a7efd10472db2d392353d807180/users/08abc209f6cd4b62b34cad53c8ba4c54/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:19:02.274775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ed63-1021-4d25-82c4-37f1a5f69e28 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:02.275296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ed63-1021-4d25-82c4-37f1a5f69e28 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:02.275296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ed63-1021-4d25-82c4-37f1a5f69e28 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:02.276671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 641/1274] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:19:02.281642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82a4e-0da5-4f86-bd08-e99955c89bbe None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:02.282015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82a4e-0da5-4f86-bd08-e99955c89bbe None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:02.282193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f82a4e-0da5-4f86-bd08-e99955c89bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:02.300020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-b7f82a4e-0da5-4f86-bd08-e99955c89bbe None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:19:02.301226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b7f82a4e-0da5-4f86-bd08-e99955c89bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:19:02.365106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 634/1275] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:19:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:19:02.388781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ed63-1021-4d25-82c4-37f1a5f69e28 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:02.489461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ed63-1021-4d25-82c4-37f1a5f69e28 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:02.508233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:02.527300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879a20a-dd70-46a3-8021-8d5b158f0795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:02.528319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879a20a-dd70-46a3-8021-8d5b158f0795 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:02.528603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879a20a-dd70-46a3-8021-8d5b158f0795 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:02.528872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879a20a-dd70-46a3-8021-8d5b158f0795 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:02.542060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879a20a-dd70-46a3-8021-8d5b158f0795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-0879a20a-dd70-46a3-8021-8d5b158f0795 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 11 16:19:02.546705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:19:02.547886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:02.548866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 642/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Aug 11 16:19:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:19:02.554752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82a4e-0da5-4f86-bd08-e99955c89bbe None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:02.651909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:02.677610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f82a4e-0da5-4f86-bd08-e99955c89bbe None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:02.720906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:02.721538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:02.721815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:02.722219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:02.722932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:02.723236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:02.724658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:02.733783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 635/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Aug 11 16:19:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:02.741831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879a20a-dd70-46a3-8021-8d5b158f0795 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:02.854763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879a20a-dd70-46a3-8021-8d5b158f0795 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:02.894902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:02.895861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:02.896275 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:02.896552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:02.906778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:02.908206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:19:02.922473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:19:02.924469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:19:02.925394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:19:02.925394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:19:02.925394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:02.925394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 643/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:19:02.930379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:03.043022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7ece2-2088-4706-97cb-1ab5457b8e91 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:03.080591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:03.081186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:03.081465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:03.081934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:03.082956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:03.083259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:03.084645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:03.098617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 636/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:19:03.104341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:03.136056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:03.212855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f80b3-5c0a-489f-8446-6b2615444cd8 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:03.251883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:03.252670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:03.252943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:03.253207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:03.260680 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:03.262097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 11 16:19:03.272723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:19:03.273867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:03.274843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 644/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:19:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:19:03.279601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:03.283907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:03.400621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85986c7-cfa5-4736-a13c-5f715319232f None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:03.434442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:03.435037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:03.435309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:03.435575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:03.436319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:03.436623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:03.438022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:03.455533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 637/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Aug 11 16:19:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:03.461288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:03.549857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:03.559218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87ae50-67b9-4e32-9ec4-3566762e1c49 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:03.591582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d2072-c1c1-4377-890b-1d4f93f56ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:03.592193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d2072-c1c1-4377-890b-1d4f93f56ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:03.592466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d2072-c1c1-4377-890b-1d4f93f56ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:03.592731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d2072-c1c1-4377-890b-1d4f93f56ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:03.659157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d2072-c1c1-4377-890b-1d4f93f56ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:03.660865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d2072-c1c1-4377-890b-1d4f93f56ce4 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:03.746637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:03.757447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 645/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Aug 11 16:19:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 11 16:19:03.763217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:03.826482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:03.864260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29a5b5-59dd-438d-ab10-189f8cac6a93 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:03.896561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:03.897170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:03.897455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:03.897725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:03.907769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:03.909232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 11 16:19:03.923895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:19:03.925052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 11 16:19:03.925957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 11 16:19:03.925957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 11 16:19:03.925957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:19:03.925957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 638/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:19:03.930625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d2072-c1c1-4377-890b-1d4f93f56ce4 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:04.024608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d2072-c1c1-4377-890b-1d4f93f56ce4 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:04.070730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829202-2568-4000-83ca-03e900c24095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:04.071339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829202-2568-4000-83ca-03e900c24095 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:04.071616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829202-2568-4000-83ca-03e900c24095 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:04.071887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829202-2568-4000-83ca-03e900c24095 None admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:04.072571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829202-2568-4000-83ca-03e900c24095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:04.072872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829202-2568-4000-83ca-03e900c24095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:04.074249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829202-2568-4000-83ca-03e900c24095 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:04.085265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 646/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Aug 11 16:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:04.090775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:04.188185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad278c-6d59-48b5-b36c-c99e3339a8e7 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:04.201659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:04.222781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:04.223395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:04.223670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:04.223938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:04.230346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:04.231778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 11 16:19:04.243595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:19:04.244683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:19:04.245831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 639/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Aug 11 16:19:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:19:04.253070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829202-2568-4000-83ca-03e900c24095 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:04.302580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:04.351960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829202-2568-4000-83ca-03e900c24095 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:04.383293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3714110-0040-479a-821c-9338a18716b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:04.383879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3714110-0040-479a-821c-9338a18716b2 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:04.384177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3714110-0040-479a-821c-9338a18716b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:04.384449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3714110-0040-479a-821c-9338a18716b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:04.385130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3714110-0040-479a-821c-9338a18716b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:04.385428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3714110-0040-479a-821c-9338a18716b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:04.386844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3714110-0040-479a-821c-9338a18716b2 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:04.398743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 647/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Aug 11 16:19:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:04.404748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:04.500805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358920a-3e13-4bdb-8f8b-5c7a7b6d7218 None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:04.533062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79702956-324e-436c-b79a-dfab140dbaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:04.533644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79702956-324e-436c-b79a-dfab140dbaed None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:04.533916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79702956-324e-436c-b79a-dfab140dbaed None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:04.534182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79702956-324e-436c-b79a-dfab140dbaed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:04.534887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79702956-324e-436c-b79a-dfab140dbaed None admin] RBAC: Adding query filter params (role.id=29fd031edd2b4f65aed8942853150f92, scope.project.id=2d968a7efd10472db2d392353d807180, user.id=08abc209f6cd4b62b34cad53c8ba4c54) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:04.535186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79702956-324e-436c-b79a-dfab140dbaed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:04.536621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79702956-324e-436c-b79a-dfab140dbaed None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:04.542076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 640/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Aug 11 16:19:04 2022] GET /identity/v3/role_assignments?user.id=08abc209f6cd4b62b34cad53c8ba4c54&scope.project.id=2d968a7efd10472db2d392353d807180&role.id=29fd031edd2b4f65aed8942853150f92 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:06.549597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe355d2-1a8c-446f-9c5a-45dc3e11d646 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:06.550150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe355d2-1a8c-446f-9c5a-45dc3e11d646 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:06.550150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe355d2-1a8c-446f-9c5a-45dc3e11d646 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:06.551411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 648/1288] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:19:06.556450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181fa1-090c-4bb2-892a-c345a17c148e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:06.556846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181fa1-090c-4bb2-892a-c345a17c148e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:06.557002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81181fa1-090c-4bb2-892a-c345a17c148e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:06.575470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-81181fa1-090c-4bb2-892a-c345a17c148e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:19:06.576701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-81181fa1-090c-4bb2-892a-c345a17c148e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:19:06.604858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:06.643904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 641/1289] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:19:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:19:06.669267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe355d2-1a8c-446f-9c5a-45dc3e11d646 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:06.773599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe355d2-1a8c-446f-9c5a-45dc3e11d646 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:06.783837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:06.806459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:06.807079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:06.807364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:06.807636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] PATH_INFO: `/v3/services/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:06.808242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:06.809591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:19:06.818828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:19:06.820080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:06.821050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 649/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:19:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:19:06.825943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181fa1-090c-4bb2-892a-c345a17c148e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:06.921782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81181fa1-090c-4bb2-892a-c345a17c148e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:06.955638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea5915-b642-4cb3-b37b-e9c03acadb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:06.956260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea5915-b642-4cb3-b37b-e9c03acadb1c None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:06.956542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea5915-b642-4cb3-b37b-e9c03acadb1c None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:06.956815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea5915-b642-4cb3-b37b-e9c03acadb1c None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:06.957442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea5915-b642-4cb3-b37b-e9c03acadb1c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:06.957743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea5915-b642-4cb3-b37b-e9c03acadb1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:06.959090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea5915-b642-4cb3-b37b-e9c03acadb1c None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:06.963718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 642/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Aug 11 16:19:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:06.969304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:07.073443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e77548-f703-4aee-816e-5b6651f578cd None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:07.106012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0f1ae-c7d3-41a4-b3fc-2fb40ec8177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:07.106610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f1ae-c7d3-41a4-b3fc-2fb40ec8177f None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:07.106890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f1ae-c7d3-41a4-b3fc-2fb40ec8177f None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:07.107162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0f1ae-c7d3-41a4-b3fc-2fb40ec8177f None admin] PATH_INFO: `/v3/services` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:07.107784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0f1ae-c7d3-41a4-b3fc-2fb40ec8177f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:07.108086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0f1ae-c7d3-41a4-b3fc-2fb40ec8177f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:07.109466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0f1ae-c7d3-41a4-b3fc-2fb40ec8177f None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:07.114347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 650/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Aug 11 16:19:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:09.100687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ffc3-b68b-430a-bac5-bfdc66b5ca95 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:09.101158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ffc3-b68b-430a-bac5-bfdc66b5ca95 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:09.101259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8ffc3-b68b-430a-bac5-bfdc66b5ca95 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:09.102540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 643/1293] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:19:09.107529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d89c-1ad7-4f54-b7d9-e3070a8e922d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:09.107962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d89c-1ad7-4f54-b7d9-e3070a8e922d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:09.108056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d89c-1ad7-4f54-b7d9-e3070a8e922d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:09.125821 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7fb9d89c-1ad7-4f54-b7d9-e3070a8e922d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:19:09.127595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7fb9d89c-1ad7-4f54-b7d9-e3070a8e922d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:19:09.156079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:09.192680 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 651/1294] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:19:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:19:09.216089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ffc3-b68b-430a-bac5-bfdc66b5ca95 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:09.246509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:09.314656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8ffc3-b68b-430a-bac5-bfdc66b5ca95 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:09.347157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e1563-fbc7-43be-a858-b302edc7f5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:09.347747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e1563-fbc7-43be-a858-b302edc7f5d3 None admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:09.348020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e1563-fbc7-43be-a858-b302edc7f5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:09.348315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e1563-fbc7-43be-a858-b302edc7f5d3 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:09.348938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e1563-fbc7-43be-a858-b302edc7f5d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:09.350278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e1563-fbc7-43be-a858-b302edc7f5d3 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:09.364245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 644/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Aug 11 16:19:09 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:19:11.431908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a63464-fc26-49a6-9cbf-8919ffd0c976 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:11.432452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a63464-fc26-49a6-9cbf-8919ffd0c976 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:11.432452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a63464-fc26-49a6-9cbf-8919ffd0c976 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:11.433744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 652/1296] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:19:11.439490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbae75d-77fc-4495-b14a-1ee3b0863fc8 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:11.439888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbae75d-77fc-4495-b14a-1ee3b0863fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:11.440034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbae75d-77fc-4495-b14a-1ee3b0863fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:11.458609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5dbae75d-77fc-4495-b14a-1ee3b0863fc8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:19:11.459786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5dbae75d-77fc-4495-b14a-1ee3b0863fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:19:11.529538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 645/1297] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:19:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:19:11.551352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a63464-fc26-49a6-9cbf-8919ffd0c976 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:11.652210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a63464-fc26-49a6-9cbf-8919ffd0c976 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:11.692649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:11.693547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:11.693831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:11.694083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:11.694658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:11.696009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:19:11.705020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:19:11.706068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:11.706947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 653/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:19:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:19:11.713070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbae75d-77fc-4495-b14a-1ee3b0863fc8 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:11.734040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:11.814479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbae75d-77fc-4495-b14a-1ee3b0863fc8 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:11.847931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:11.848549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:11.848823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:11.849087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:11.849704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:11.849997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:11.851347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:11.856242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 646/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Aug 11 16:19:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:11.862954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:11.893627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:11.959506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056184b0-478b-4a7c-95b7-4274e6f3b052 None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:11.991296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bc7af-ae9b-40df-ab68-4f4ba93e85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:11.991876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bc7af-ae9b-40df-ab68-4f4ba93e85bb None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:11.992175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bc7af-ae9b-40df-ab68-4f4ba93e85bb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:11.992444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bc7af-ae9b-40df-ab68-4f4ba93e85bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:11.993128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bc7af-ae9b-40df-ab68-4f4ba93e85bb None admin] RBAC: Adding query filter params (interface=public, service_id=c5bc31be6c794ad3bf9964f398707f2e, region_id=RegionOne) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:11.993427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bc7af-ae9b-40df-ab68-4f4ba93e85bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:11.994771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15bc7af-ae9b-40df-ab68-4f4ba93e85bb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:11.999530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 654/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Aug 11 16:19:11 2022] GET /identity/v3/endpoints?service_id=c5bc31be6c794ad3bf9964f398707f2e&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:12.005775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:12.086238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:12.104704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36d85-ea25-4c75-9290-f96fda8ecddd None admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:12.137662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f6eb8-7fc8-46cf-92b8-fb2b5445e0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:12.138251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f6eb8-7fc8-46cf-92b8-fb2b5445e0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:12.138528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f6eb8-7fc8-46cf-92b8-fb2b5445e0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:12.138798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f6eb8-7fc8-46cf-92b8-fb2b5445e0a0 None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:12.139430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f6eb8-7fc8-46cf-92b8-fb2b5445e0a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:12.140805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f6eb8-7fc8-46cf-92b8-fb2b5445e0a0 None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:12.147464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 647/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Aug 11 16:19:12 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:19:14.156946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64a46b-ed3a-4258-98c1-55e59c4d332e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:14.157481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64a46b-ed3a-4258-98c1-55e59c4d332e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:14.157481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64a46b-ed3a-4258-98c1-55e59c4d332e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:14.158815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 655/1302] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:19:14.163973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ca774-dc9e-4685-b2e7-942900ebaa95 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:14.164423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ca774-dc9e-4685-b2e7-942900ebaa95 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:14.164556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ca774-dc9e-4685-b2e7-942900ebaa95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:14.182643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c42ca774-dc9e-4685-b2e7-942900ebaa95 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:19:14.183831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c42ca774-dc9e-4685-b2e7-942900ebaa95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:19:14.249299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 648/1303] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:19:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 11 16:19:14.278839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64a46b-ed3a-4258-98c1-55e59c4d332e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:14.371383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:14.378824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64a46b-ed3a-4258-98c1-55e59c4d332e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:14.411496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:14.412131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:14.412401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:14.412667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] PATH_INFO: `/v3/services/placement` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:14.413236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:14.414733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:19:14.423823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:19:14.425063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:19:14.425982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 656/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Aug 11 16:19:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 11 16:19:14.431001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ca774-dc9e-4685-b2e7-942900ebaa95 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:14.504491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:14.526552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ca774-dc9e-4685-b2e7-942900ebaa95 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:14.560150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb7d21-0ed7-4bc3-b6f8-1a3f7a5f260b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:14.560748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb7d21-0ed7-4bc3-b6f8-1a3f7a5f260b None admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:14.561027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb7d21-0ed7-4bc3-b6f8-1a3f7a5f260b None admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:14.561296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb7d21-0ed7-4bc3-b6f8-1a3f7a5f260b None admin] PATH_INFO: `/v3/services` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:14.561918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb7d21-0ed7-4bc3-b6f8-1a3f7a5f260b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:19:14.562220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb7d21-0ed7-4bc3-b6f8-1a3f7a5f260b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:14.563622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb7d21-0ed7-4bc3-b6f8-1a3f7a5f260b None admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:14.568947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 649/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Aug 11 16:19:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:19:14.574593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:19:14.670310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899742e-ba1c-4eea-880b-9f78046c09df None admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:19:14.702395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19451201-c256-4e5d-a951-d55e056178fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:19:14.702977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451201-c256-4e5d-a951-d55e056178fb None admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:19:14.703254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451201-c256-4e5d-a951-d55e056178fb None admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:19:14.703518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451201-c256-4e5d-a951-d55e056178fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:19:14.704879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19451201-c256-4e5d-a951-d55e056178fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:19:14.706220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19451201-c256-4e5d-a951-d55e056178fb None admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:19:14.725230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 657/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:19:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:19:24.565758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=30 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:19:24.721086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:18.549830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cee541-294f-4ae4-926d-3ee1c031e2c8 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:18.550415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cee541-294f-4ae4-926d-3ee1c031e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:18.550415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cee541-294f-4ae4-926d-3ee1c031e2c8 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:18.556921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726099c1-6d71-4895-aa84-c7ff268a1fb8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:18.557306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726099c1-6d71-4895-aa84-c7ff268a1fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:18.557513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726099c1-6d71-4895-aa84-c7ff268a1fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:18.562330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 650/1307] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:18.643737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-726099c1-6d71-4895-aa84-c7ff268a1fb8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:18.644986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-726099c1-6d71-4895-aa84-c7ff268a1fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:18.776573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 658/1308] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:20:18.817344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782fc1b-5957-4e54-a5c1-1ead25dfbe87 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:18.817636 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782fc1b-5957-4e54-a5c1-1ead25dfbe87 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:18.817912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782fc1b-5957-4e54-a5c1-1ead25dfbe87 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:18.819615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 651/1309] 10.222.0.43 () {54 vars in 965 bytes} [Thu Aug 11 16:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:18.826530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8738a5-1852-4a15-9bf9-9d5c072bedf3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:18.826790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8738a5-1852-4a15-9bf9-9d5c072bedf3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:18.827111 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8738a5-1852-4a15-9bf9-9d5c072bedf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:18.920130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cc8738a5-1852-4a15-9bf9-9d5c072bedf3 None None] MFA Rules not processed for user `123d33bb6caa464aba4f9be6b0fb7656`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:18.921938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cc8738a5-1852-4a15-9bf9-9d5c072bedf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:18.945670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:19.006849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 659/1310] 10.222.0.43 () {58 vars in 1062 bytes} [Thu Aug 11 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:19.015830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782fc1b-5957-4e54-a5c1-1ead25dfbe87 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:19.018378 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f782fc1b-5957-4e54-a5c1-1ead25dfbe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:19.140457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782fc1b-5957-4e54-a5c1-1ead25dfbe87 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:19.227154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac4eff-85cb-43ce-b715-313fb47c6dfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:19.227689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4eff-85cb-43ce-b715-313fb47c6dfa service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:19.227972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4eff-85cb-43ce-b715-313fb47c6dfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:19.228277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4eff-85cb-43ce-b715-313fb47c6dfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:19.229991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-91ac4eff-85cb-43ce-b715-313fb47c6dfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:19.292067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:19.354929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac4eff-85cb-43ce-b715-313fb47c6dfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:19.356409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac4eff-85cb-43ce-b715-313fb47c6dfa service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:19.377226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 652/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:21.700386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08a19-cc4f-4496-8e6f-a9a3754e48c6 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:21.700910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08a19-cc4f-4496-8e6f-a9a3754e48c6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:21.700910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d08a19-cc4f-4496-8e6f-a9a3754e48c6 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:21.702291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 660/1312] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:21.707728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69df74-aa25-425d-ae1c-163d97c88dd0 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:21.708194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69df74-aa25-425d-ae1c-163d97c88dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:21.708291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69df74-aa25-425d-ae1c-163d97c88dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:21.777574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-af69df74-aa25-425d-ae1c-163d97c88dd0 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:21.778780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-af69df74-aa25-425d-ae1c-163d97c88dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:21.878807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 653/1313] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:20:21.917026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d08a19-cc4f-4496-8e6f-a9a3754e48c6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:21.918647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-05d08a19-cc4f-4496-8e6f-a9a3754e48c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:22.022905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d08a19-cc4f-4496-8e6f-a9a3754e48c6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:22.092880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:22.096149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de3bf-cd50-4e49-a4e2-e94f9d7e171e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:22.096974 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de3bf-cd50-4e49-a4e2-e94f9d7e171e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:22.097420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de3bf-cd50-4e49-a4e2-e94f9d7e171e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:22.097891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de3bf-cd50-4e49-a4e2-e94f9d7e171e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:22.099535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef0de3bf-cd50-4e49-a4e2-e94f9d7e171e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:22.207658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0de3bf-cd50-4e49-a4e2-e94f9d7e171e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:22.209056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0de3bf-cd50-4e49-a4e2-e94f9d7e171e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:22.228540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 661/1314] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:24.558269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f618cc-4b87-4327-97bb-269205f187cd None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:24.558799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f618cc-4b87-4327-97bb-269205f187cd None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:24.558799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f618cc-4b87-4327-97bb-269205f187cd None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:24.560170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 654/1315] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:24.566033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3e3bf-9642-4f37-b695-4718062bfc40 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:24.566456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3e3bf-9642-4f37-b695-4718062bfc40 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:24.566589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3e3bf-9642-4f37-b695-4718062bfc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:24.752282 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-12d3e3bf-9642-4f37-b695-4718062bfc40 None None] MFA Rules not processed for user `0898fbca99fc478e9cab823c368a904f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:24.753498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-12d3e3bf-9642-4f37-b695-4718062bfc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:24.804927 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:24.849679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 662/1316] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:24.901469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640b66c-f014-44de-ae6a-fd3842401539 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:24.901751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640b66c-f014-44de-ae6a-fd3842401539 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:24.902024 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640b66c-f014-44de-ae6a-fd3842401539 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:24.903377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 655/1317] 10.222.0.43 () {54 vars in 965 bytes} [Thu Aug 11 16:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:24.909846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82b567-9faa-4975-a86d-7b3d4d31c949 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:24.910133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82b567-9faa-4975-a86d-7b3d4d31c949 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:24.910472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82b567-9faa-4975-a86d-7b3d4d31c949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:24.984644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5f82b567-9faa-4975-a86d-7b3d4d31c949 None None] MFA Rules not processed for user `123d33bb6caa464aba4f9be6b0fb7656`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:24.986444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5f82b567-9faa-4975-a86d-7b3d4d31c949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:25.060865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 663/1318] 10.222.0.43 () {58 vars in 1062 bytes} [Thu Aug 11 16:20:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:20:25.070408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640b66c-f014-44de-ae6a-fd3842401539 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:25.072267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8640b66c-f014-44de-ae6a-fd3842401539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:25.081911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:25.173703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640b66c-f014-44de-ae6a-fd3842401539 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:25.244715 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca15736-61b4-4a3f-912b-59da92d8572e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:25.245318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca15736-61b4-4a3f-912b-59da92d8572e service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:25.245598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca15736-61b4-4a3f-912b-59da92d8572e service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:25.245872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca15736-61b4-4a3f-912b-59da92d8572e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:25.247484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4ca15736-61b4-4a3f-912b-59da92d8572e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:25.307908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:25.354684 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca15736-61b4-4a3f-912b-59da92d8572e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:25.356119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca15736-61b4-4a3f-912b-59da92d8572e service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:25.376462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 656/1319] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:25 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:28.334386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3e6ec-86c8-40a9-bb24-7fa7b38389de None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:28.334909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3e6ec-86c8-40a9-bb24-7fa7b38389de None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:28.334909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3e6ec-86c8-40a9-bb24-7fa7b38389de None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:28.336306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 664/1320] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:28.341339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360759-dae0-4dc4-8a70-7546d7996d20 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:28.341759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360759-dae0-4dc4-8a70-7546d7996d20 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:28.341913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360759-dae0-4dc4-8a70-7546d7996d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:28.407040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-13360759-dae0-4dc4-8a70-7546d7996d20 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:28.408506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-13360759-dae0-4dc4-8a70-7546d7996d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:28.503685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 657/1321] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:28.542108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3e6ec-86c8-40a9-bb24-7fa7b38389de None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:28.543630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a4b3e6ec-86c8-40a9-bb24-7fa7b38389de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:28.592888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:28.653679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3e6ec-86c8-40a9-bb24-7fa7b38389de None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:28.727360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfb917-6c98-498f-a73b-b5c96f8fdb98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:28.727966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfb917-6c98-498f-a73b-b5c96f8fdb98 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:28.728279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfb917-6c98-498f-a73b-b5c96f8fdb98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:28.728551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfb917-6c98-498f-a73b-b5c96f8fdb98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:28.730139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30cfb917-6c98-498f-a73b-b5c96f8fdb98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:28.827626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:28.842767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfb917-6c98-498f-a73b-b5c96f8fdb98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:28.844263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfb917-6c98-498f-a73b-b5c96f8fdb98 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:28.863600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 665/1322] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:31.577263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe7c8-ec57-4f28-bda2-8fca438c956a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:31.577786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe7c8-ec57-4f28-bda2-8fca438c956a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:31.577786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fe7c8-ec57-4f28-bda2-8fca438c956a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:31.579145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 658/1323] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:31.585141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153195-2943-48dc-bb36-dd938746dcb4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:31.585552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153195-2943-48dc-bb36-dd938746dcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:31.585696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5153195-2943-48dc-bb36-dd938746dcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:31.653755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e5153195-2943-48dc-bb36-dd938746dcb4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:31.655003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e5153195-2943-48dc-bb36-dd938746dcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:31.762593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 666/1324] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:31.801014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fe7c8-ec57-4f28-bda2-8fca438c956a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:31.802573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f44fe7c8-ec57-4f28-bda2-8fca438c956a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:31.823342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:31.911554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fe7c8-ec57-4f28-bda2-8fca438c956a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:31.984466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5de43-3c13-44bc-b1f4-9028997c21f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:31.985074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5de43-3c13-44bc-b1f4-9028997c21f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:31.985348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5de43-3c13-44bc-b1f4-9028997c21f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:31.985615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5de43-3c13-44bc-b1f4-9028997c21f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:31.987800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-eab5de43-3c13-44bc-b1f4-9028997c21f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:32.062707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:32.101515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5de43-3c13-44bc-b1f4-9028997c21f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:32.102948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5de43-3c13-44bc-b1f4-9028997c21f0 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:32.121550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 659/1325] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:34.211926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf507af-2ff7-4452-baac-4f1aea546513 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:34.212472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf507af-2ff7-4452-baac-4f1aea546513 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:34.212472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf507af-2ff7-4452-baac-4f1aea546513 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:34.213738 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 667/1326] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:34.218878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619a63-952a-4a77-a1ce-1c4453a2b7ef None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:34.219291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619a63-952a-4a77-a1ce-1c4453a2b7ef None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:34.219420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f619a63-952a-4a77-a1ce-1c4453a2b7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:34.286175 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1f619a63-952a-4a77-a1ce-1c4453a2b7ef None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:34.287374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1f619a63-952a-4a77-a1ce-1c4453a2b7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:34.383297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 660/1327] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:34.419911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf507af-2ff7-4452-baac-4f1aea546513 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:34.421471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dbf507af-2ff7-4452-baac-4f1aea546513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:34.487715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:34.536740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf507af-2ff7-4452-baac-4f1aea546513 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:34.616642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e7700-1b17-4569-8a43-72b792e42fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:34.617248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e7700-1b17-4569-8a43-72b792e42fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:34.617520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e7700-1b17-4569-8a43-72b792e42fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:34.617777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e7700-1b17-4569-8a43-72b792e42fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:34.619349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f22e7700-1b17-4569-8a43-72b792e42fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:34.730239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:34.732700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e7700-1b17-4569-8a43-72b792e42fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:34.734437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e7700-1b17-4569-8a43-72b792e42fa7 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:34.753961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 668/1328] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:36.810762 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5ac6-ec4e-4900-baeb-1bd6c6f6d2e9 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:36.811338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5ac6-ec4e-4900-baeb-1bd6c6f6d2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:36.811338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d5ac6-ec4e-4900-baeb-1bd6c6f6d2e9 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:36.812711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 661/1329] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:36.818484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2935-f749-4acb-98db-1b90b051a7b3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:36.818929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2935-f749-4acb-98db-1b90b051a7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:36.819035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2935-f749-4acb-98db-1b90b051a7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:36.887222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-deac2935-f749-4acb-98db-1b90b051a7b3 None None] MFA Rules not processed for user `0898fbca99fc478e9cab823c368a904f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:36.888389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-deac2935-f749-4acb-98db-1b90b051a7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:36.989998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 669/1330] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:37.029544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d5ac6-ec4e-4900-baeb-1bd6c6f6d2e9 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:37.031164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-702d5ac6-ec4e-4900-baeb-1bd6c6f6d2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:37.062366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:37.136408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d5ac6-ec4e-4900-baeb-1bd6c6f6d2e9 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:37.209508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab2c04-c14a-485c-831f-a1c18c9e3abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:37.210116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab2c04-c14a-485c-831f-a1c18c9e3abe service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:37.210389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab2c04-c14a-485c-831f-a1c18c9e3abe service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:37.210656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab2c04-c14a-485c-831f-a1c18c9e3abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:37.212311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0dab2c04-c14a-485c-831f-a1c18c9e3abe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:37.293429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:37.326464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab2c04-c14a-485c-831f-a1c18c9e3abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:37.327885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab2c04-c14a-485c-831f-a1c18c9e3abe service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:37.346373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 662/1331] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:39.655443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe9479-9cf3-4f8c-9816-d75fd8dfc80c None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:39.656013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe9479-9cf3-4f8c-9816-d75fd8dfc80c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:39.656013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe9479-9cf3-4f8c-9816-d75fd8dfc80c None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:39.657374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 670/1332] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:39.663060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795ee9-a012-476e-a1a9-f6c75559ffb4 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:39.663491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795ee9-a012-476e-a1a9-f6c75559ffb4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:39.663679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795ee9-a012-476e-a1a9-f6c75559ffb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:39.729759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d4795ee9-a012-476e-a1a9-f6c75559ffb4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:39.730887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d4795ee9-a012-476e-a1a9-f6c75559ffb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:39.827141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 663/1333] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:39.864129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe9479-9cf3-4f8c-9816-d75fd8dfc80c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:39.865653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b1fe9479-9cf3-4f8c-9816-d75fd8dfc80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:39.938388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:39.978536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe9479-9cf3-4f8c-9816-d75fd8dfc80c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:40.056496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a95bf-e412-42d4-9ded-2d788a18e724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:40.057109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a95bf-e412-42d4-9ded-2d788a18e724 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:40.057383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a95bf-e412-42d4-9ded-2d788a18e724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:40.057650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a95bf-e412-42d4-9ded-2d788a18e724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:40.059246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9a4a95bf-e412-42d4-9ded-2d788a18e724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:40.169906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a95bf-e412-42d4-9ded-2d788a18e724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:40.171328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a95bf-e412-42d4-9ded-2d788a18e724 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:40.181065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:40.191326 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 671/1334] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:42.235719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16558fd8-9abb-4681-bcc7-0bfc883bfc9e None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:42.236700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16558fd8-9abb-4681-bcc7-0bfc883bfc9e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:42.236700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16558fd8-9abb-4681-bcc7-0bfc883bfc9e None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:42.237709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 664/1335] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:42.242882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed903d95-6053-4f4a-8d1d-bcd15a471e77 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:42.243299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed903d95-6053-4f4a-8d1d-bcd15a471e77 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:42.243429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed903d95-6053-4f4a-8d1d-bcd15a471e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:42.307283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ed903d95-6053-4f4a-8d1d-bcd15a471e77 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:42.308509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ed903d95-6053-4f4a-8d1d-bcd15a471e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:42.406176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 672/1336] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:20:42.447344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16558fd8-9abb-4681-bcc7-0bfc883bfc9e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:42.448918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-16558fd8-9abb-4681-bcc7-0bfc883bfc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:42.488909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:42.552814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16558fd8-9abb-4681-bcc7-0bfc883bfc9e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:42.624452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619fdb6c-9797-495c-8691-185d67025a53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:42.625047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fdb6c-9797-495c-8691-185d67025a53 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:42.625327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fdb6c-9797-495c-8691-185d67025a53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:42.625594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619fdb6c-9797-495c-8691-185d67025a53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:42.628036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-619fdb6c-9797-495c-8691-185d67025a53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:42.710395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:42.737808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619fdb6c-9797-495c-8691-185d67025a53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:42.739293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619fdb6c-9797-495c-8691-185d67025a53 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:42.756947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 665/1337] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:45.084787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4bcad-752f-4af2-a7fd-b97844a49ec0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:45.085296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4bcad-752f-4af2-a7fd-b97844a49ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:45.085296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4bcad-752f-4af2-a7fd-b97844a49ec0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:45.086656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 673/1338] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:45.092151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438e29-58e9-40b3-9c40-ffa305ea9b35 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:45.092659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438e29-58e9-40b3-9c40-ffa305ea9b35 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:45.092757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1438e29-58e9-40b3-9c40-ffa305ea9b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:45.158002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d1438e29-58e9-40b3-9c40-ffa305ea9b35 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:45.159217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d1438e29-58e9-40b3-9c40-ffa305ea9b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:45.253903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 666/1339] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:45.289973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4bcad-752f-4af2-a7fd-b97844a49ec0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:45.291494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f0f4bcad-752f-4af2-a7fd-b97844a49ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:45.366663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:45.394229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4bcad-752f-4af2-a7fd-b97844a49ec0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:45.464621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8834526-5716-43fe-81a4-94f3bdba4876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:45.465218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8834526-5716-43fe-81a4-94f3bdba4876 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:45.465493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8834526-5716-43fe-81a4-94f3bdba4876 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:45.465756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8834526-5716-43fe-81a4-94f3bdba4876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:45.467331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f8834526-5716-43fe-81a4-94f3bdba4876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:45.577577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8834526-5716-43fe-81a4-94f3bdba4876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:45.579003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8834526-5716-43fe-81a4-94f3bdba4876 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:45.597308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 674/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:49.677322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb36a-4a8c-4d5f-9a1c-312d1b547f57 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:49.677882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb36a-4a8c-4d5f-9a1c-312d1b547f57 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:49.677882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb36a-4a8c-4d5f-9a1c-312d1b547f57 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:49.679212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 667/1341] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:49.684710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bda0f-0265-4dd0-ab89-c070f9da3394 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:49.685175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bda0f-0265-4dd0-ab89-c070f9da3394 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:49.685276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bda0f-0265-4dd0-ab89-c070f9da3394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:49.690948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:49.752903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f85bda0f-0265-4dd0-ab89-c070f9da3394 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:49.754065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f85bda0f-0265-4dd0-ab89-c070f9da3394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:49.852621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 675/1342] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:49.892684 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bb36a-4a8c-4d5f-9a1c-312d1b547f57 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:49.894231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-576bb36a-4a8c-4d5f-9a1c-312d1b547f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:49.944800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:50.002797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bb36a-4a8c-4d5f-9a1c-312d1b547f57 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:50.077258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03d8bc-f3cf-453e-96a9-d7de5b145558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:50.077857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03d8bc-f3cf-453e-96a9-d7de5b145558 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:50.078132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03d8bc-f3cf-453e-96a9-d7de5b145558 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:50.078396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03d8bc-f3cf-453e-96a9-d7de5b145558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:50.080559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ff03d8bc-f3cf-453e-96a9-d7de5b145558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:50.172349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:50.191222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03d8bc-f3cf-453e-96a9-d7de5b145558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:50.193210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03d8bc-f3cf-453e-96a9-d7de5b145558 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:50.212188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 668/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:52.912211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e544a0-b7f3-4d1b-9c97-6489dee6f088 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:52.912672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e544a0-b7f3-4d1b-9c97-6489dee6f088 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:52.912771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e544a0-b7f3-4d1b-9c97-6489dee6f088 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:52.914144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 676/1344] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:52.919401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb5eeb-860b-44d9-b9a2-b0f23042b747 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:52.919857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb5eeb-860b-44d9-b9a2-b0f23042b747 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:52.919966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb5eeb-860b-44d9-b9a2-b0f23042b747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:52.999464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3dcb5eeb-860b-44d9-b9a2-b0f23042b747 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:53.000700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3dcb5eeb-860b-44d9-b9a2-b0f23042b747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:53.107430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 669/1345] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:53.144980 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e544a0-b7f3-4d1b-9c97-6489dee6f088 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:53.146522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e9e544a0-b7f3-4d1b-9c97-6489dee6f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:53.234824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:53.253030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e544a0-b7f3-4d1b-9c97-6489dee6f088 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:53.326281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13133df7-17ce-4163-b351-4ce1447620c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:53.326884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13133df7-17ce-4163-b351-4ce1447620c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:53.327160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13133df7-17ce-4163-b351-4ce1447620c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:53.327427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13133df7-17ce-4163-b351-4ce1447620c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:53.329043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-13133df7-17ce-4163-b351-4ce1447620c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:53.435280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13133df7-17ce-4163-b351-4ce1447620c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:53.436755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13133df7-17ce-4163-b351-4ce1447620c8 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:53.457412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 677/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:20:57.740714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffac4f-85a7-48b4-a4a1-7807aa83db47 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:57.741283 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffac4f-85a7-48b4-a4a1-7807aa83db47 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:57.741384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffac4f-85a7-48b4-a4a1-7807aa83db47 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:57.742806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 670/1347] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:20:57.748625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71559eb8-b322-4a11-a3c0-c399f2f62071 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:57.748625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71559eb8-b322-4a11-a3c0-c399f2f62071 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:57.749167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71559eb8-b322-4a11-a3c0-c399f2f62071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:57.765945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:57.821573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-71559eb8-b322-4a11-a3c0-c399f2f62071 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:20:57.823072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-71559eb8-b322-4a11-a3c0-c399f2f62071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:57.925349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 678/1348] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:20:57.964361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffac4f-85a7-48b4-a4a1-7807aa83db47 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:20:57.965977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a9ffac4f-85a7-48b4-a4a1-7807aa83db47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:58.036182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:58.091407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffac4f-85a7-48b4-a4a1-7807aa83db47 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:20:58.173517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aaa3b-0ee6-441e-9701-bcc9a61bbe02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:20:58.174023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aaa3b-0ee6-441e-9701-bcc9a61bbe02 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:20:58.174299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aaa3b-0ee6-441e-9701-bcc9a61bbe02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:20:58.174563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aaa3b-0ee6-441e-9701-bcc9a61bbe02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:20:58.176283 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-432aaa3b-0ee6-441e-9701-bcc9a61bbe02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:20:58.302520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:20:58.308366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aaa3b-0ee6-441e-9701-bcc9a61bbe02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:20:58.310246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aaa3b-0ee6-441e-9701-bcc9a61bbe02 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:20:58.331523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 671/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:03.271580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22cbb3-de09-4627-853a-1023dab454dd None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:03.271580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22cbb3-de09-4627-853a-1023dab454dd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:03.271580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22cbb3-de09-4627-853a-1023dab454dd None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:03.275414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 679/1350] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:21:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:03.281721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8fec5-cfcc-4e41-bf3e-a573c2de36b9 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:03.282176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8fec5-cfcc-4e41-bf3e-a573c2de36b9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:03.282281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8fec5-cfcc-4e41-bf3e-a573c2de36b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:03.480224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-60a8fec5-cfcc-4e41-bf3e-a573c2de36b9 None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:03.480224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-60a8fec5-cfcc-4e41-bf3e-a573c2de36b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:03.649683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 672/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:21:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:07.923112 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=59 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:07.955533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62b62-bd62-48ec-9852-6c7972aec2c7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:07.955877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62b62-bd62-48ec-9852-6c7972aec2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:07.956170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62b62-bd62-48ec-9852-6c7972aec2c7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:07.957624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 680/1352] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:07.962629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e007b5c-b691-406e-b672-2ea7bc5b8b3e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:07.963145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e007b5c-b691-406e-b672-2ea7bc5b8b3e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:07.963397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e007b5c-b691-406e-b672-2ea7bc5b8b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:08.032276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:08.059641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3e007b5c-b691-406e-b672-2ea7bc5b8b3e None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:08.060903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3e007b5c-b691-406e-b672-2ea7bc5b8b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:08.182911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 673/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:21:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:09.355678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b3aa-8124-4ece-aaa4-3858b7a7dc98 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:09.356288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b3aa-8124-4ece-aaa4-3858b7a7dc98 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:09.356288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b3aa-8124-4ece-aaa4-3858b7a7dc98 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:09.357884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 681/1354] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:09.365530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594e7f-ae5f-4c7c-8731-ca0cf9fd6582 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:09.365988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594e7f-ae5f-4c7c-8731-ca0cf9fd6582 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:09.366079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44594e7f-ae5f-4c7c-8731-ca0cf9fd6582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:09.440782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-44594e7f-ae5f-4c7c-8731-ca0cf9fd6582 None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:09.441970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-44594e7f-ae5f-4c7c-8731-ca0cf9fd6582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:09.483766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:09.550575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 674/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:21:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:13.101795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4385b9-7091-4400-acf7-214e2f3e60aa None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:13.102759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4385b9-7091-4400-acf7-214e2f3e60aa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:13.103043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4385b9-7091-4400-acf7-214e2f3e60aa None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:13.104753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 682/1356] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:13.109784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb6c00-d241-4245-ac61-b6541bdf6dc7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:13.110258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb6c00-d241-4245-ac61-b6541bdf6dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:13.110355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb6c00-d241-4245-ac61-b6541bdf6dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:13.200343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-aadb6c00-d241-4245-ac61-b6541bdf6dc7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:13.201499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-aadb6c00-d241-4245-ac61-b6541bdf6dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:13.307486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 675/1357] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:13.362391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9e173-18a4-4fad-a1a2-178f26dda312 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:13.362970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9e173-18a4-4fad-a1a2-178f26dda312 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:13.363344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9e173-18a4-4fad-a1a2-178f26dda312 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:13.364829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 683/1358] 10.222.0.43 () {54 vars in 962 bytes} [Thu Aug 11 16:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:13.372631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ba6f-1560-433a-9822-0e164e233415 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:13.372923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ba6f-1560-433a-9822-0e164e233415 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:13.373210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381ba6f-1560-433a-9822-0e164e233415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:13.394773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:13.464225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-8381ba6f-1560-433a-9822-0e164e233415 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:13.465424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8381ba6f-1560-433a-9822-0e164e233415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:13.597023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 676/1359] 10.222.0.43 () {58 vars in 1059 bytes} [Thu Aug 11 16:21:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:13.604731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9e173-18a4-4fad-a1a2-178f26dda312 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:13.606437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e7c9e173-18a4-4fad-a1a2-178f26dda312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:13.770365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9e173-18a4-4fad-a1a2-178f26dda312 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:13.857393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:13.902520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30986048-dfe2-439c-9039-f018ef2f6eff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:13.903139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30986048-dfe2-439c-9039-f018ef2f6eff service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:13.903414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30986048-dfe2-439c-9039-f018ef2f6eff service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:13.903681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30986048-dfe2-439c-9039-f018ef2f6eff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:13.905564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30986048-dfe2-439c-9039-f018ef2f6eff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:14.002312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30986048-dfe2-439c-9039-f018ef2f6eff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:14.003894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30986048-dfe2-439c-9039-f018ef2f6eff service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:14.024288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 684/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:16.252379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3eda4-a708-4615-a7f3-d37043e513bb None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:16.252881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3eda4-a708-4615-a7f3-d37043e513bb None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:16.253137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3eda4-a708-4615-a7f3-d37043e513bb None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:16.256930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 677/1361] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:16.259420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8859b6-4dcd-4605-ab89-40b10a2538dd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:16.259825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8859b6-4dcd-4605-ab89-40b10a2538dd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:16.259991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8859b6-4dcd-4605-ab89-40b10a2538dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:16.335179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ab8859b6-4dcd-4605-ab89-40b10a2538dd None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:16.336413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ab8859b6-4dcd-4605-ab89-40b10a2538dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:16.377067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:16.442112 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 685/1362] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:16.487052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3eda4-a708-4615-a7f3-d37043e513bb None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:16.488846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-75a3eda4-a708-4615-a7f3-d37043e513bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:16.567349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:16.670006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3eda4-a708-4615-a7f3-d37043e513bb None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:16.805185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d8e72-21b7-4fb6-9b13-d7a6cac47c75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:16.806236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8e72-21b7-4fb6-9b13-d7a6cac47c75 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:16.806236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8e72-21b7-4fb6-9b13-d7a6cac47c75 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:16.806692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d8e72-21b7-4fb6-9b13-d7a6cac47c75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:16.808394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a17d8e72-21b7-4fb6-9b13-d7a6cac47c75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:16.833296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:16.907765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d8e72-21b7-4fb6-9b13-d7a6cac47c75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:16.910076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d8e72-21b7-4fb6-9b13-d7a6cac47c75 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:16.929288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 678/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:19.312227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae19665-d4f0-40e4-973c-b0776b57ab91 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:19.312776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae19665-d4f0-40e4-973c-b0776b57ab91 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:19.312776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae19665-d4f0-40e4-973c-b0776b57ab91 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:19.314965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 686/1364] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:19.320469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33d163-5299-4f44-bf2c-386c09736afb None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:19.320938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33d163-5299-4f44-bf2c-386c09736afb None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:19.321063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33d163-5299-4f44-bf2c-386c09736afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:19.400588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5a33d163-5299-4f44-bf2c-386c09736afb None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:19.401835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5a33d163-5299-4f44-bf2c-386c09736afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:19.510462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:19.524495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 679/1365] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:19.560695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97afd04-88a4-4e08-a655-2ba61b7eece1 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:19.561033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97afd04-88a4-4e08-a655-2ba61b7eece1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:19.561377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97afd04-88a4-4e08-a655-2ba61b7eece1 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:19.562999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 687/1366] 10.222.0.43 () {54 vars in 962 bytes} [Thu Aug 11 16:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:19.569230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb59fe-0a0f-4bc3-a2be-39e4e4350ad9 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:19.569761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb59fe-0a0f-4bc3-a2be-39e4e4350ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:19.570221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb59fe-0a0f-4bc3-a2be-39e4e4350ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:19.656590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-6aeb59fe-0a0f-4bc3-a2be-39e4e4350ad9 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:19.658781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6aeb59fe-0a0f-4bc3-a2be-39e4e4350ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:19.785162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 680/1367] 10.222.0.43 () {58 vars in 1059 bytes} [Thu Aug 11 16:21:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:19.792889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97afd04-88a4-4e08-a655-2ba61b7eece1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:19.794385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d97afd04-88a4-4e08-a655-2ba61b7eece1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:19.952795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97afd04-88a4-4e08-a655-2ba61b7eece1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:19.975199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:20.076396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48c806-4a41-4aff-abcf-2bb07e84d7c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:20.077009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c806-4a41-4aff-abcf-2bb07e84d7c6 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:20.077288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c806-4a41-4aff-abcf-2bb07e84d7c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:20.077554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48c806-4a41-4aff-abcf-2bb07e84d7c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:20.079138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6b48c806-4a41-4aff-abcf-2bb07e84d7c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:20.171097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48c806-4a41-4aff-abcf-2bb07e84d7c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:20.172753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48c806-4a41-4aff-abcf-2bb07e84d7c6 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:20.192573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 688/1368] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:21.374127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23823515-6633-4af3-8403-a2b8cfe51624 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:21.374618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23823515-6633-4af3-8403-a2b8cfe51624 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:21.374618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23823515-6633-4af3-8403-a2b8cfe51624 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:21.376082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 681/1369] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:21.381634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4b88d-2066-4162-b240-17c38fc61b49 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:21.382679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4b88d-2066-4162-b240-17c38fc61b49 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:21.383291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4b88d-2066-4162-b240-17c38fc61b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:21.422867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:21.463687 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fce4b88d-2066-4162-b240-17c38fc61b49 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:21.465339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fce4b88d-2066-4162-b240-17c38fc61b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:21.580133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 689/1370] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:21.916260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635e6cc-ba8b-4fd0-92f0-d8b635425de7 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:21.916260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635e6cc-ba8b-4fd0-92f0-d8b635425de7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:21.916595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635e6cc-ba8b-4fd0-92f0-d8b635425de7 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:21.917972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 682/1371] 10.222.0.43 () {54 vars in 964 bytes} [Thu Aug 11 16:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:21.922778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92ece-3871-4aba-9299-abc191491c99 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:21.923166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92ece-3871-4aba-9299-abc191491c99 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:21.924168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb92ece-3871-4aba-9299-abc191491c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:22.011308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9bb92ece-3871-4aba-9299-abc191491c99 None None] MFA Rules not processed for user `89757c5dc48942259ffdd67e0cd19ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:22.012710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9bb92ece-3871-4aba-9299-abc191491c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:22.067134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:22.100716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 690/1372] 10.222.0.43 () {58 vars in 1061 bytes} [Thu Aug 11 16:21:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:21:22.108126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635e6cc-ba8b-4fd0-92f0-d8b635425de7 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:22.109662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6635e6cc-ba8b-4fd0-92f0-d8b635425de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:22.162963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:22.217936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635e6cc-ba8b-4fd0-92f0-d8b635425de7 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:22.296883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:22.297773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:22.298200 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:22.299311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:22.301372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:22.331647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1689ccc-2cc9-442a-b36d-955f57dc0d3d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:22.331932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1689ccc-2cc9-442a-b36d-955f57dc0d3d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:22.332237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1689ccc-2cc9-442a-b36d-955f57dc0d3d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:22.333590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 691/1373] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:22.338450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8e7b5-86fd-46b5-ae68-e1b7d3ae5960 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:22.338723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8e7b5-86fd-46b5-ae68-e1b7d3ae5960 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:22.338996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8e7b5-86fd-46b5-ae68-e1b7d3ae5960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:22.412565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6cf8e7b5-86fd-46b5-ae68-e1b7d3ae5960 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:22.413860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6cf8e7b5-86fd-46b5-ae68-e1b7d3ae5960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:22.415968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:22.433202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:22.434748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:22.456277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 683/1374] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:22.531462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 692/1375] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:21:22.578273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:22.578273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:22.692877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ec488-1640-40fd-916f-6ae8e9a04e42 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:22.821103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:22.841590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65020d-9433-4140-a92a-bfe605c92b03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:22.842193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65020d-9433-4140-a92a-bfe605c92b03 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:22.842466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65020d-9433-4140-a92a-bfe605c92b03 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:22.842733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65020d-9433-4140-a92a-bfe605c92b03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:22.844362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4c65020d-9433-4140-a92a-bfe605c92b03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:22.937658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65020d-9433-4140-a92a-bfe605c92b03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:22.940175 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65020d-9433-4140-a92a-bfe605c92b03 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:22.958939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 684/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:25.029079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664a167-98b7-4e16-b7e9-25819df2a3d2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:25.030051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664a167-98b7-4e16-b7e9-25819df2a3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:25.030541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664a167-98b7-4e16-b7e9-25819df2a3d2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:25.032601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 693/1377] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:25.038753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4837ee-b1cd-4e02-bf86-1422f846de9f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:25.039234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4837ee-b1cd-4e02-bf86-1422f846de9f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:25.039319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4837ee-b1cd-4e02-bf86-1422f846de9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:25.123694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-8e4837ee-b1cd-4e02-bf86-1422f846de9f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:25.124929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8e4837ee-b1cd-4e02-bf86-1422f846de9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:25.183486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:25.224444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 685/1378] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:25.268569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664a167-98b7-4e16-b7e9-25819df2a3d2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:25.270079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c664a167-98b7-4e16-b7e9-25819df2a3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:25.331267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:25.432339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664a167-98b7-4e16-b7e9-25819df2a3d2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:25.583273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd4d61-4d00-43a9-8734-afe807e132ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:25.583812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd4d61-4d00-43a9-8734-afe807e132ef service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:25.584089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd4d61-4d00-43a9-8734-afe807e132ef service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:25.584384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd4d61-4d00-43a9-8734-afe807e132ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:25.585983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-79bd4d61-4d00-43a9-8734-afe807e132ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:25.597730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:25.676212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd4d61-4d00-43a9-8734-afe807e132ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:25.677786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd4d61-4d00-43a9-8734-afe807e132ef service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:25.696191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 694/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:27.760154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363a62-c06f-4c0e-9e3d-64889dae6930 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:27.760736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363a62-c06f-4c0e-9e3d-64889dae6930 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:27.760736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363a62-c06f-4c0e-9e3d-64889dae6930 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:27.762159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 686/1380] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:27.767230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6a6d-b7c1-48f3-b915-e7bae11afabc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:27.769434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6a6d-b7c1-48f3-b915-e7bae11afabc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:27.770029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6a6d-b7c1-48f3-b915-e7bae11afabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:27.846605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-22fa6a6d-b7c1-48f3-b915-e7bae11afabc None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:27.848053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-22fa6a6d-b7c1-48f3-b915-e7bae11afabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:27.941865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:27.969246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 695/1381] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:28.003926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c363a62-c06f-4c0e-9e3d-64889dae6930 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:28.005480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1c363a62-c06f-4c0e-9e3d-64889dae6930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:28.178869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c363a62-c06f-4c0e-9e3d-64889dae6930 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:28.229729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:28.306364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1336a3-5d7e-4036-8717-2bfd6a31de29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:28.307171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1336a3-5d7e-4036-8717-2bfd6a31de29 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:28.307667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1336a3-5d7e-4036-8717-2bfd6a31de29 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:28.307978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1336a3-5d7e-4036-8717-2bfd6a31de29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:28.309695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3d1336a3-5d7e-4036-8717-2bfd6a31de29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:28.411821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1336a3-5d7e-4036-8717-2bfd6a31de29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:28.413456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1336a3-5d7e-4036-8717-2bfd6a31de29 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:28.432475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 687/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:30.683776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9708e2f-f261-40b0-8d02-1723ca169a0f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:30.684330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9708e2f-f261-40b0-8d02-1723ca169a0f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:30.684330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9708e2f-f261-40b0-8d02-1723ca169a0f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:30.685685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 696/1383] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:30.692561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0d46-868f-4b6d-97ad-b0f5c80460a1 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:30.693032 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0d46-868f-4b6d-97ad-b0f5c80460a1 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:30.693131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a0d46-868f-4b6d-97ad-b0f5c80460a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:30.762701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-825a0d46-868f-4b6d-97ad-b0f5c80460a1 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:30.764555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-825a0d46-868f-4b6d-97ad-b0f5c80460a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:30.767247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:30.868801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 688/1384] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:30.915912 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9708e2f-f261-40b0-8d02-1723ca169a0f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:30.917586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d9708e2f-f261-40b0-8d02-1723ca169a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:31.070764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9708e2f-f261-40b0-8d02-1723ca169a0f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:31.137327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:31.187703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d91b5e-2dec-4903-add1-95bca05ee2dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:31.188368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91b5e-2dec-4903-add1-95bca05ee2dd service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:31.188660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91b5e-2dec-4903-add1-95bca05ee2dd service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:31.188905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d91b5e-2dec-4903-add1-95bca05ee2dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:31.191240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-09d91b5e-2dec-4903-add1-95bca05ee2dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:31.283096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d91b5e-2dec-4903-add1-95bca05ee2dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:31.284727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d91b5e-2dec-4903-add1-95bca05ee2dd service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:31.303305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 697/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:33.309633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098eaf7-cbe6-4a10-8928-b3e8bae07c68 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:33.310156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098eaf7-cbe6-4a10-8928-b3e8bae07c68 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:33.310156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098eaf7-cbe6-4a10-8928-b3e8bae07c68 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:33.311440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 689/1386] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:33.317072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17906244-5127-47cd-9c3f-3b69cf88c557 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:33.317552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17906244-5127-47cd-9c3f-3b69cf88c557 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:33.317686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17906244-5127-47cd-9c3f-3b69cf88c557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:33.385628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-17906244-5127-47cd-9c3f-3b69cf88c557 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:33.386847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-17906244-5127-47cd-9c3f-3b69cf88c557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:33.418247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:33.504156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 698/1387] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:33.548451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098eaf7-cbe6-4a10-8928-b3e8bae07c68 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:33.552755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5098eaf7-cbe6-4a10-8928-b3e8bae07c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:33.686273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:33.703747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098eaf7-cbe6-4a10-8928-b3e8bae07c68 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:33.842412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b85ca2-fd98-47ab-956c-3b2ccb476267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:33.843012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b85ca2-fd98-47ab-956c-3b2ccb476267 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:33.843343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b85ca2-fd98-47ab-956c-3b2ccb476267 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:33.843719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b85ca2-fd98-47ab-956c-3b2ccb476267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:33.845300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-09b85ca2-fd98-47ab-956c-3b2ccb476267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:33.947345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b85ca2-fd98-47ab-956c-3b2ccb476267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:33.949901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b85ca2-fd98-47ab-956c-3b2ccb476267 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:33.957122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:33.973220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 690/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:36.025571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9e2ef-cc7a-43b8-90ee-d31d0d8bf95b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:36.026129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9e2ef-cc7a-43b8-90ee-d31d0d8bf95b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:36.026129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9e2ef-cc7a-43b8-90ee-d31d0d8bf95b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:36.027473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 699/1389] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:21:36.032757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753bd6-5042-4ae1-8bbc-1154f70e4d77 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:36.033222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753bd6-5042-4ae1-8bbc-1154f70e4d77 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:36.033352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753bd6-5042-4ae1-8bbc-1154f70e4d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:36.101136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f0753bd6-5042-4ae1-8bbc-1154f70e4d77 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:36.102352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f0753bd6-5042-4ae1-8bbc-1154f70e4d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:36.201203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 691/1390] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:36.245795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9e2ef-cc7a-43b8-90ee-d31d0d8bf95b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:36.247323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0dc9e2ef-cc7a-43b8-90ee-d31d0d8bf95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:36.394769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9e2ef-cc7a-43b8-90ee-d31d0d8bf95b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:36.403723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:36.506613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ccace-6bba-4974-8c87-797589c012bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:36.507140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ccace-6bba-4974-8c87-797589c012bc service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:36.507419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ccace-6bba-4974-8c87-797589c012bc service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:36.507688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ccace-6bba-4974-8c87-797589c012bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:36.509349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-498ccace-6bba-4974-8c87-797589c012bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:36.598304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ccace-6bba-4974-8c87-797589c012bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:36.599892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ccace-6bba-4974-8c87-797589c012bc service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:36.618446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 700/1391] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:37.581667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7916b-350b-4627-8a8d-038d992b300a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:37.582225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7916b-350b-4627-8a8d-038d992b300a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:37.582225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7916b-350b-4627-8a8d-038d992b300a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:37.583539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 692/1392] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:37.588800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8614-c481-419f-97de-b4a0a79f20c6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:37.589232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8614-c481-419f-97de-b4a0a79f20c6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:37.589355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b8614-c481-419f-97de-b4a0a79f20c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:37.604745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:37.655627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8b7b8614-c481-419f-97de-b4a0a79f20c6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:37.656866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8b7b8614-c481-419f-97de-b4a0a79f20c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:37.758514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 701/1393] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:37.816727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed9b6-5566-4e09-87c1-dc22f0c50e7d None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:37.817075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed9b6-5566-4e09-87c1-dc22f0c50e7d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:37.817402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ed9b6-5566-4e09-87c1-dc22f0c50e7d None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:37.818785 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 693/1394] 10.222.0.43 () {54 vars in 967 bytes} [Thu Aug 11 16:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:37.824490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e7d55-9b75-4564-9dd0-9258d66b8cae None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:37.824844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e7d55-9b75-4564-9dd0-9258d66b8cae None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:37.825182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e7d55-9b75-4564-9dd0-9258d66b8cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:37.900954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e40e7d55-9b75-4564-9dd0-9258d66b8cae None None] MFA Rules not processed for user `f7a2237c18934cb2aa3699a5776874ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:37.902061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e40e7d55-9b75-4564-9dd0-9258d66b8cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:37.936937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:37.980431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 702/1395] 10.222.0.43 () {58 vars in 1064 bytes} [Thu Aug 11 16:21:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:37.988710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed9b6-5566-4e09-87c1-dc22f0c50e7d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:37.990240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ca8ed9b6-5566-4e09-87c1-dc22f0c50e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:38.068697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:38.100305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ed9b6-5566-4e09-87c1-dc22f0c50e7d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:38.175127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe9009-921b-4fbf-a5f2-4b5fbe073c9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7a2237c18934cb2aa3699a5776874ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:38.175647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe9009-921b-4fbf-a5f2-4b5fbe073c9f service glance] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:38.175927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe9009-921b-4fbf-a5f2-4b5fbe073c9f service glance] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:38.176250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe9009-921b-4fbf-a5f2-4b5fbe073c9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:38.177868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-adbe9009-921b-4fbf-a5f2-4b5fbe073c9f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:38.291582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe9009-921b-4fbf-a5f2-4b5fbe073c9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:38.293180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe9009-921b-4fbf-a5f2-4b5fbe073c9f service glance] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:38.309195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:38.314399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 694/1396] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:21:38.703396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7d8d7-0ed0-4b83-97d2-ce51e4b21174 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:38.704007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7d8d7-0ed0-4b83-97d2-ce51e4b21174 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:38.704147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7d8d7-0ed0-4b83-97d2-ce51e4b21174 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:38.709881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 703/1397] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:38.715905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ee28b-5688-4162-a68a-7ba9a5bde28c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:38.716470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ee28b-5688-4162-a68a-7ba9a5bde28c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:38.716470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ee28b-5688-4162-a68a-7ba9a5bde28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:38.782315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-659ee28b-5688-4162-a68a-7ba9a5bde28c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:38.784167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-659ee28b-5688-4162-a68a-7ba9a5bde28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:38.882868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 695/1398] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:21:38.930011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7d8d7-0ed0-4b83-97d2-ce51e4b21174 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:38.931560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f5f7d8d7-0ed0-4b83-97d2-ce51e4b21174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:39.070683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aa55d-383f-4639-9080-288fb6549233 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:39.070969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aa55d-383f-4639-9080-288fb6549233 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:39.071243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aa55d-383f-4639-9080-288fb6549233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:39.090946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7d8d7-0ed0-4b83-97d2-ce51e4b21174 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:39.133324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:39.139910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c10aa55d-383f-4639-9080-288fb6549233 None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:39.141155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c10aa55d-383f-4639-9080-288fb6549233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:39.170391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:39.216000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:39.216656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:39.216961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:39.217230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:39.218846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:39.235712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 696/1399] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:21:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:21:39.279007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a54500-8a49-4bcd-9f6b-a3ec79447b90 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:39.279293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a54500-8a49-4bcd-9f6b-a3ec79447b90 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:39.279941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a54500-8a49-4bcd-9f6b-a3ec79447b90 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:39.281352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 697/1400] 10.222.0.43 () {54 vars in 943 bytes} [Thu Aug 11 16:21:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:21:39.288151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54eaae-469f-4981-a40e-3546399e0dcf None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:39.288538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54eaae-469f-4981-a40e-3546399e0dcf None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:39.288890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54eaae-469f-4981-a40e-3546399e0dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:39.317803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:39.319310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:39.338093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 704/1401] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:39.372921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-db54eaae-469f-4981-a40e-3546399e0dcf None None] MFA Rules not processed for user `256cc27b87684392a22f36429b960644`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:39.372921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-db54eaae-469f-4981-a40e-3546399e0dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:39.488205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 698/1402] 10.222.0.43 () {58 vars in 1040 bytes} [Thu Aug 11 16:21:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:21:39.498436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:39.499672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:39.549092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:39.592268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4da05-2ecf-413f-bc97-5c619ef19819 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:39.698386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8de603-b466-45a6-9f3f-2e560056f8f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:39.698996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8de603-b466-45a6-9f3f-2e560056f8f9 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:39.699539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8de603-b466-45a6-9f3f-2e560056f8f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:39.699808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8de603-b466-45a6-9f3f-2e560056f8f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:39.701470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-db8de603-b466-45a6-9f3f-2e560056f8f9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:39.784339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:39.788324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8de603-b466-45a6-9f3f-2e560056f8f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:39.789910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8de603-b466-45a6-9f3f-2e560056f8f9 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:39.797916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 705/1403] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:21:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:21:41.433596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809091-edbc-4191-9a36-31c5666fbf38 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:41.434138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809091-edbc-4191-9a36-31c5666fbf38 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:41.434138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809091-edbc-4191-9a36-31c5666fbf38 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:41.441365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e41004-e8bc-494b-924f-ea16e6bf3b48 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:41.441933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e41004-e8bc-494b-924f-ea16e6bf3b48 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:41.441933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e41004-e8bc-494b-924f-ea16e6bf3b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:41.444231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 699/1404] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:41.516992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-23e41004-e8bc-494b-924f-ea16e6bf3b48 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:41.518863 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-23e41004-e8bc-494b-924f-ea16e6bf3b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:41.624336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 706/1405] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:21:41.670084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2809091-edbc-4191-9a36-31c5666fbf38 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:41.672019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e2809091-edbc-4191-9a36-31c5666fbf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:41.676076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:41.844143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2809091-edbc-4191-9a36-31c5666fbf38 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:41.935259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:41.983496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f41422-ca83-42fe-a326-7534adba9a17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:41.984125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f41422-ca83-42fe-a326-7534adba9a17 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:41.984416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f41422-ca83-42fe-a326-7534adba9a17 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:41.984687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f41422-ca83-42fe-a326-7534adba9a17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:41.986984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-75f41422-ca83-42fe-a326-7534adba9a17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:42.089871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f41422-ca83-42fe-a326-7534adba9a17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:42.091459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f41422-ca83-42fe-a326-7534adba9a17 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:42.114865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 700/1406] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:44.207034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca3f53-e918-44ff-8388-9caa10bca974 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:44.208042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca3f53-e918-44ff-8388-9caa10bca974 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:44.208455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca3f53-e918-44ff-8388-9caa10bca974 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:44.212137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 707/1407] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:21:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:21:44.215280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77889-8e73-432f-907d-2c11119354ee None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:44.215724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77889-8e73-432f-907d-2c11119354ee None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:44.215836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb77889-8e73-432f-907d-2c11119354ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:44.289220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-dcb77889-8e73-432f-907d-2c11119354ee None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:21:44.290455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-dcb77889-8e73-432f-907d-2c11119354ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:44.330799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:44.394046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 701/1408] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:21:44.442205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca3f53-e918-44ff-8388-9caa10bca974 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:21:44.443809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6cca3f53-e918-44ff-8388-9caa10bca974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:44.527561 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:44.607976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca3f53-e918-44ff-8388-9caa10bca974 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:21:44.736464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a49add-474f-4873-b240-e1fad49d958f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:21:44.737356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a49add-474f-4873-b240-e1fad49d958f service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:21:44.737634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a49add-474f-4873-b240-e1fad49d958f service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:21:44.737902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a49add-474f-4873-b240-e1fad49d958f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:21:44.739517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-42a49add-474f-4873-b240-e1fad49d958f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:21:44.778537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:44.843406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a49add-474f-4873-b240-e1fad49d958f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:21:44.845105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a49add-474f-4873-b240-e1fad49d958f service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:21:44.865528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 708/1409] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:21:54.391893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=26 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:21:54.862441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=30 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:02.457365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bee19-66a2-4071-86c2-9e3ecc438246 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:02.457890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bee19-66a2-4071-86c2-9e3ecc438246 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:02.457890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bee19-66a2-4071-86c2-9e3ecc438246 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:02.459154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 702/1410] 10.222.0.43 () {54 vars in 918 bytes} [Thu Aug 11 16:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:02.464530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0d180-8f9c-44ec-abdc-e71e094ca98a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:02.464962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0d180-8f9c-44ec-abdc-e71e094ca98a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:02.465106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0d180-8f9c-44ec-abdc-e71e094ca98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:02.540934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c4a0d180-8f9c-44ec-abdc-e71e094ca98a None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:02.542144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c4a0d180-8f9c-44ec-abdc-e71e094ca98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:02.648079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 709/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Aug 11 16:22:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:02.662939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbe1e0-f76a-4d27-ad3e-7a6a8654bfaf None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:02.663230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbe1e0-f76a-4d27-ad3e-7a6a8654bfaf None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:02.663504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbe1e0-f76a-4d27-ad3e-7a6a8654bfaf None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:02.664862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 703/1412] 10.222.0.43 () {54 vars in 918 bytes} [Thu Aug 11 16:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:02.669485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3a8a8-04d3-4b2d-bf56-ac43615b8ad9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:02.669795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3a8a8-04d3-4b2d-bf56-ac43615b8ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:02.670094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3a8a8-04d3-4b2d-bf56-ac43615b8ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:02.744398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:02.746993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-21d3a8a8-04d3-4b2d-bf56-ac43615b8ad9 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:02.748784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-21d3a8a8-04d3-4b2d-bf56-ac43615b8ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:02.864618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 710/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Aug 11 16:22:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:05.453987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b041f-67c7-4e43-801e-e00bd24cc81b None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:05.454579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b041f-67c7-4e43-801e-e00bd24cc81b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:05.454579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b041f-67c7-4e43-801e-e00bd24cc81b None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:05.456570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:05.461755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9751d8-b67c-46ab-be55-777004c7ccde None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:05.462172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9751d8-b67c-46ab-be55-777004c7ccde None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:05.462303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9751d8-b67c-46ab-be55-777004c7ccde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:05.528909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cf9751d8-b67c-46ab-be55-777004c7ccde None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:05.530107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cf9751d8-b67c-46ab-be55-777004c7ccde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:05.584560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:05.626036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:05.669821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b041f-67c7-4e43-801e-e00bd24cc81b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:05.671384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-647b041f-67c7-4e43-801e-e00bd24cc81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:05.815189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b041f-67c7-4e43-801e-e00bd24cc81b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:05.887618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:05.928166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e91795-17dc-49d7-85e9-6c2ff0a2f4ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:05.928775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91795-17dc-49d7-85e9-6c2ff0a2f4ab service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:05.929052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91795-17dc-49d7-85e9-6c2ff0a2f4ab service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:05.929319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e91795-17dc-49d7-85e9-6c2ff0a2f4ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:05.930929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f5e91795-17dc-49d7-85e9-6c2ff0a2f4ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:06.019128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e91795-17dc-49d7-85e9-6c2ff0a2f4ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:06.020781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e91795-17dc-49d7-85e9-6c2ff0a2f4ab service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:06.038692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 705/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:06.949512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff0b52-c283-4f73-a3f1-de35439dfffe None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:06.950336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff0b52-c283-4f73-a3f1-de35439dfffe None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:06.950336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff0b52-c283-4f73-a3f1-de35439dfffe None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:06.951713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:06.957535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eda24-3689-4a48-8c93-a8ac35326efc None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:06.957971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eda24-3689-4a48-8c93-a8ac35326efc None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:06.958081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eda24-3689-4a48-8c93-a8ac35326efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:07.033036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-081eda24-3689-4a48-8c93-a8ac35326efc None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:07.034374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-081eda24-3689-4a48-8c93-a8ac35326efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:07.061915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:07.132447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:07.281366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff0b52-c283-4f73-a3f1-de35439dfffe None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:07.282982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b3ff0b52-c283-4f73-a3f1-de35439dfffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:07.427182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff0b52-c283-4f73-a3f1-de35439dfffe None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:07.466840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:07.539152 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc444f23-f9a3-480f-811e-affb7d0af3dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:07.539686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444f23-f9a3-480f-811e-affb7d0af3dc service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:07.539983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444f23-f9a3-480f-811e-affb7d0af3dc service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:07.540271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc444f23-f9a3-480f-811e-affb7d0af3dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:07.541868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cc444f23-f9a3-480f-811e-affb7d0af3dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:07.628949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc444f23-f9a3-480f-811e-affb7d0af3dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:07.630511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc444f23-f9a3-480f-811e-affb7d0af3dc service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:07.648586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 713/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:08.530541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6abec1-f819-4518-b7cb-92c37cebcde6 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:08.531115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6abec1-f819-4518-b7cb-92c37cebcde6 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:08.531115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6abec1-f819-4518-b7cb-92c37cebcde6 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:08.533135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:08.538309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c264a0-e611-427c-84fe-926918fee5df None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:08.538737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c264a0-e611-427c-84fe-926918fee5df None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:08.538856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c264a0-e611-427c-84fe-926918fee5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:08.591751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:08.615508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-90c264a0-e611-427c-84fe-926918fee5df None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:08.616745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-90c264a0-e611-427c-84fe-926918fee5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:08.714559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:08.864995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6abec1-f819-4518-b7cb-92c37cebcde6 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:08.866782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2e6abec1-f819-4518-b7cb-92c37cebcde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:09.013550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6abec1-f819-4518-b7cb-92c37cebcde6 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:09.026693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:09.127660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ac132-2208-4ca3-8915-f7ea50b346e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:09.128204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ac132-2208-4ca3-8915-f7ea50b346e2 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:09.128484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ac132-2208-4ca3-8915-f7ea50b346e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:09.128758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ac132-2208-4ca3-8915-f7ea50b346e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:09.130363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a11ac132-2208-4ca3-8915-f7ea50b346e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:09.219982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ac132-2208-4ca3-8915-f7ea50b346e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:09.222095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ac132-2208-4ca3-8915-f7ea50b346e2 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:09.240030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 708/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:10.107270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd8249-1c31-4316-8b6f-a460b67b3de0 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:10.107709 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd8249-1c31-4316-8b6f-a460b67b3de0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:10.107836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd8249-1c31-4316-8b6f-a460b67b3de0 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:10.109230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:22:10.114586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68797b-5195-4b37-9381-191241816e76 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:10.114980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68797b-5195-4b37-9381-191241816e76 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:10.115147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68797b-5195-4b37-9381-191241816e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:10.134538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:10.191250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ee68797b-5195-4b37-9381-191241816e76 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:10.192523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ee68797b-5195-4b37-9381-191241816e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:10.287722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:10.440169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd8249-1c31-4316-8b6f-a460b67b3de0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:10.441672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2fcd8249-1c31-4316-8b6f-a460b67b3de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:10.550433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:10.586597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd8249-1c31-4316-8b6f-a460b67b3de0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:10.696206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd1bd4-fa58-4373-8c80-58268c7d1894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:10.696809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd1bd4-fa58-4373-8c80-58268c7d1894 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:10.697085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd1bd4-fa58-4373-8c80-58268c7d1894 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:10.697352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd1bd4-fa58-4373-8c80-58268c7d1894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:10.698932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d1cd1bd4-fa58-4373-8c80-58268c7d1894 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:10.774629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:10.788788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd1bd4-fa58-4373-8c80-58268c7d1894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:10.790404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd1bd4-fa58-4373-8c80-58268c7d1894 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:10.808456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 716/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:22:11.667937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326a3c8-41b7-4a78-9a9f-ca55e65c63ba None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:11.668528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326a3c8-41b7-4a78-9a9f-ca55e65c63ba None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:11.668528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326a3c8-41b7-4a78-9a9f-ca55e65c63ba None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:11.669828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:22:11.674704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cede69-a32e-4f73-8124-0322a33542b2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:11.675109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cede69-a32e-4f73-8124-0322a33542b2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:11.675246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cede69-a32e-4f73-8124-0322a33542b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:11.753674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-16cede69-a32e-4f73-8124-0322a33542b2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:11.754890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-16cede69-a32e-4f73-8124-0322a33542b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:11.850708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:12.002897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326a3c8-41b7-4a78-9a9f-ca55e65c63ba None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:12.004455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6326a3c8-41b7-4a78-9a9f-ca55e65c63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:12.088933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:12.151385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326a3c8-41b7-4a78-9a9f-ca55e65c63ba None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:12.262566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145822fe-fbf9-4045-99b0-9852d15de0df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:12.263086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145822fe-fbf9-4045-99b0-9852d15de0df service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:12.263359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145822fe-fbf9-4045-99b0-9852d15de0df service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:12.263626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145822fe-fbf9-4045-99b0-9852d15de0df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:12.265261 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-145822fe-fbf9-4045-99b0-9852d15de0df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:12.312228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:12.354026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145822fe-fbf9-4045-99b0-9852d15de0df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:12.355625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145822fe-fbf9-4045-99b0-9852d15de0df service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:12.373858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 711/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:13.187116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8838cb3-e66b-480e-8c67-470b1ca836f7 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:13.187688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8838cb3-e66b-480e-8c67-470b1ca836f7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:13.187688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8838cb3-e66b-480e-8c67-470b1ca836f7 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:13.189520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:13.194455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62412-e114-4465-947d-1d16e99ba004 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:13.194886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62412-e114-4465-947d-1d16e99ba004 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:13.195003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd62412-e114-4465-947d-1d16e99ba004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:13.271372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ccd62412-e114-4465-947d-1d16e99ba004 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:13.272611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ccd62412-e114-4465-947d-1d16e99ba004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:13.368303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:13.522145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8838cb3-e66b-480e-8c67-470b1ca836f7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:13.523668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e8838cb3-e66b-480e-8c67-470b1ca836f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:13.572247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:13.671140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8838cb3-e66b-480e-8c67-470b1ca836f7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:13.787794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd5e95-2d27-4dff-9b15-3695bab15fec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:13.788434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd5e95-2d27-4dff-9b15-3695bab15fec service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:13.788708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd5e95-2d27-4dff-9b15-3695bab15fec service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:13.788972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd5e95-2d27-4dff-9b15-3695bab15fec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:13.790645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9dbd5e95-2d27-4dff-9b15-3695bab15fec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:13.797757 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:13.881007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd5e95-2d27-4dff-9b15-3695bab15fec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:13.882588 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd5e95-2d27-4dff-9b15-3695bab15fec service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:13.900162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 719/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:14.704030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9011549-e798-44ee-9db9-a1f3c4ccce5d None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:14.704601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9011549-e798-44ee-9db9-a1f3c4ccce5d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:14.704601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9011549-e798-44ee-9db9-a1f3c4ccce5d None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:14.705932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:14.710776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6039c-d3dd-4629-8cec-f8576c734ecc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:14.711189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6039c-d3dd-4629-8cec-f8576c734ecc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:14.711340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6039c-d3dd-4629-8cec-f8576c734ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:14.787170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a3e6039c-d3dd-4629-8cec-f8576c734ecc None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:14.789329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a3e6039c-d3dd-4629-8cec-f8576c734ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:14.856785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:14.886799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:15.038398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9011549-e798-44ee-9db9-a1f3c4ccce5d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:15.039940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b9011549-e798-44ee-9db9-a1f3c4ccce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:15.058088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:15.186366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9011549-e798-44ee-9db9-a1f3c4ccce5d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:15.275721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:15.302717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f014861-35b0-4b93-ae4c-198a50183d87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:15.303318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f014861-35b0-4b93-ae4c-198a50183d87 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:15.303592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f014861-35b0-4b93-ae4c-198a50183d87 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:15.303855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f014861-35b0-4b93-ae4c-198a50183d87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:15.305479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1f014861-35b0-4b93-ae4c-198a50183d87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:15.393555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f014861-35b0-4b93-ae4c-198a50183d87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:15.395156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f014861-35b0-4b93-ae4c-198a50183d87 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:15.413764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 714/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:16.210509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19649a22-ac41-49ca-bd83-a98aecd8bb4e None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:16.211060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19649a22-ac41-49ca-bd83-a98aecd8bb4e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:16.211060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19649a22-ac41-49ca-bd83-a98aecd8bb4e None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:16.212436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:16.217245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101fbe3-c631-447c-bcf7-9f99eecc265d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:16.217682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101fbe3-c631-447c-bcf7-9f99eecc265d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:16.217793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101fbe3-c631-447c-bcf7-9f99eecc265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:16.292536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-0101fbe3-c631-447c-bcf7-9f99eecc265d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:16.293738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0101fbe3-c631-447c-bcf7-9f99eecc265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:16.454925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:16.509059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:16.661075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19649a22-ac41-49ca-bd83-a98aecd8bb4e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:16.662581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-19649a22-ac41-49ca-bd83-a98aecd8bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:16.805239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19649a22-ac41-49ca-bd83-a98aecd8bb4e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:16.858887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:16.917777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f06260-2e71-4c02-b65c-16d56c3cc3e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:16.918384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f06260-2e71-4c02-b65c-16d56c3cc3e1 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:16.918665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f06260-2e71-4c02-b65c-16d56c3cc3e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:16.918937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f06260-2e71-4c02-b65c-16d56c3cc3e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:16.920567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-07f06260-2e71-4c02-b65c-16d56c3cc3e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:17.009222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f06260-2e71-4c02-b65c-16d56c3cc3e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:17.010801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f06260-2e71-4c02-b65c-16d56c3cc3e1 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:17.028446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 722/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:17.834564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061badff-bfa6-40e8-a73e-7dc1784edf38 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:17.835116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061badff-bfa6-40e8-a73e-7dc1784edf38 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:17.835116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061badff-bfa6-40e8-a73e-7dc1784edf38 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:17.836443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:17.841247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff6cb2-f92b-4142-9d29-cc3c001b7d29 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:17.841664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff6cb2-f92b-4142-9d29-cc3c001b7d29 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:17.841799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff6cb2-f92b-4142-9d29-cc3c001b7d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:17.918270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a3ff6cb2-f92b-4142-9d29-cc3c001b7d29 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:17.919470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a3ff6cb2-f92b-4142-9d29-cc3c001b7d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:17.922014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:18.018144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:18.168949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061badff-bfa6-40e8-a73e-7dc1784edf38 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:18.170420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-061badff-bfa6-40e8-a73e-7dc1784edf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:18.313296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061badff-bfa6-40e8-a73e-7dc1784edf38 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:18.340164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:18.428954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eed40-adcb-4e32-9d36-ef400701f82f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:18.429481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eed40-adcb-4e32-9d36-ef400701f82f service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:18.429754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eed40-adcb-4e32-9d36-ef400701f82f service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:18.430020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eed40-adcb-4e32-9d36-ef400701f82f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:18.431652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c35eed40-adcb-4e32-9d36-ef400701f82f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:18.518670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eed40-adcb-4e32-9d36-ef400701f82f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:18.520270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eed40-adcb-4e32-9d36-ef400701f82f service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:18.538405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 717/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:20.449509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec5031-68d4-4ce4-9bd4-db103beaaa73 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:20.450042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec5031-68d4-4ce4-9bd4-db103beaaa73 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:20.450169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec5031-68d4-4ce4-9bd4-db103beaaa73 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:20.451453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:20.457792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417817cb-ba94-4816-8f54-04bd7662575e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:20.458186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417817cb-ba94-4816-8f54-04bd7662575e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:20.458324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417817cb-ba94-4816-8f54-04bd7662575e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:20.498643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:20.526448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-417817cb-ba94-4816-8f54-04bd7662575e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:20.527668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-417817cb-ba94-4816-8f54-04bd7662575e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:20.623154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:20.662226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec5031-68d4-4ce4-9bd4-db103beaaa73 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:20.663845 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-59ec5031-68d4-4ce4-9bd4-db103beaaa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:20.769235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec5031-68d4-4ce4-9bd4-db103beaaa73 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:20.801376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:20.842472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcec59-7f08-447e-bf5d-57a264f87e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:20.843181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcec59-7f08-447e-bf5d-57a264f87e27 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:20.843459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcec59-7f08-447e-bf5d-57a264f87e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:20.843729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcec59-7f08-447e-bf5d-57a264f87e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:20.845354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4afcec59-7f08-447e-bf5d-57a264f87e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:20.953741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcec59-7f08-447e-bf5d-57a264f87e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:20.955164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcec59-7f08-447e-bf5d-57a264f87e27 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:20.973781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 725/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:23.211366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbc69e-7d50-4bc7-897f-1e7a7498fee5 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:23.211929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbc69e-7d50-4bc7-897f-1e7a7498fee5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:23.211929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbc69e-7d50-4bc7-897f-1e7a7498fee5 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:23.213309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:23.219137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cdb95-fc47-4fe7-a23f-f6203f75c01e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:23.219556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cdb95-fc47-4fe7-a23f-f6203f75c01e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:23.219690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cdb95-fc47-4fe7-a23f-f6203f75c01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:23.286103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a26cdb95-fc47-4fe7-a23f-f6203f75c01e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:23.287304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a26cdb95-fc47-4fe7-a23f-f6203f75c01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:23.300907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:23.383219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:23.421924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbc69e-7d50-4bc7-897f-1e7a7498fee5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:23.423528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-18bbc69e-7d50-4bc7-897f-1e7a7498fee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:23.524368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbc69e-7d50-4bc7-897f-1e7a7498fee5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:23.537278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:23.597925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8b83b-36c0-4010-8ebe-0766bbba06bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:23.598525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8b83b-36c0-4010-8ebe-0766bbba06bc service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:23.598807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8b83b-36c0-4010-8ebe-0766bbba06bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:23.599075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8b83b-36c0-4010-8ebe-0766bbba06bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:23.600699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9de8b83b-36c0-4010-8ebe-0766bbba06bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:23.709727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8b83b-36c0-4010-8ebe-0766bbba06bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:23.711080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8b83b-36c0-4010-8ebe-0766bbba06bc service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:23.729408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 720/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:26.025190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308061ba-198c-48c8-b787-c70a48b990b2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:26.025700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308061ba-198c-48c8-b787-c70a48b990b2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:26.025700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308061ba-198c-48c8-b787-c70a48b990b2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:26.027068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:26.033176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc267a-a5ac-4a79-a430-b954e1c3ee12 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:26.033617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc267a-a5ac-4a79-a430-b954e1c3ee12 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:26.033750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc267a-a5ac-4a79-a430-b954e1c3ee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:26.081753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:26.099930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-dabc267a-a5ac-4a79-a430-b954e1c3ee12 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:26.101158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-dabc267a-a5ac-4a79-a430-b954e1c3ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:26.198695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:26.239239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308061ba-198c-48c8-b787-c70a48b990b2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:26.240840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-308061ba-198c-48c8-b787-c70a48b990b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:26.346760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308061ba-198c-48c8-b787-c70a48b990b2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:26.387470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:26.422275 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2666b-55ad-4764-af3a-8ab495075f2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:26.422876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2666b-55ad-4764-af3a-8ab495075f2e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:26.423156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2666b-55ad-4764-af3a-8ab495075f2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:26.423425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2666b-55ad-4764-af3a-8ab495075f2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:26.425053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-45a2666b-55ad-4764-af3a-8ab495075f2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:26.534978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2666b-55ad-4764-af3a-8ab495075f2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:26.536352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2666b-55ad-4764-af3a-8ab495075f2e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:26.554770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 728/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:28.707766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a44b1-75c5-480e-83a2-ed1e2a725942 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:28.708363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a44b1-75c5-480e-83a2-ed1e2a725942 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:28.708363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a44b1-75c5-480e-83a2-ed1e2a725942 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:28.709674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:28.714935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c216d-e41a-4531-abb8-cb8c4a8320c4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:28.715317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c216d-e41a-4531-abb8-cb8c4a8320c4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:28.715475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c216d-e41a-4531-abb8-cb8c4a8320c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:28.782774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9d4c216d-e41a-4531-abb8-cb8c4a8320c4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:28.783996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9d4c216d-e41a-4531-abb8-cb8c4a8320c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:28.804977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:28.882103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:28.921985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a44b1-75c5-480e-83a2-ed1e2a725942 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:28.923527 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1e3a44b1-75c5-480e-83a2-ed1e2a725942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:29.025039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a44b1-75c5-480e-83a2-ed1e2a725942 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:29.045521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:29.101543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c50f98-1a20-48b7-b647-a08fb7e4a47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:29.102142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c50f98-1a20-48b7-b647-a08fb7e4a47b service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:29.102415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c50f98-1a20-48b7-b647-a08fb7e4a47b service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:29.102679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c50f98-1a20-48b7-b647-a08fb7e4a47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:29.104292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-53c50f98-1a20-48b7-b647-a08fb7e4a47b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:29.216594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c50f98-1a20-48b7-b647-a08fb7e4a47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:29.218547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c50f98-1a20-48b7-b647-a08fb7e4a47b service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:29.237996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 723/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:31.337478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2f09-2bfa-4db4-85bb-fb0ed97a22d2 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:31.338022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2f09-2bfa-4db4-85bb-fb0ed97a22d2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:31.338022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e2f09-2bfa-4db4-85bb-fb0ed97a22d2 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:31.339328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:31.345486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110f414-ec67-4625-b01b-f23a1d628c95 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:31.345994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110f414-ec67-4625-b01b-f23a1d628c95 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:31.346116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110f414-ec67-4625-b01b-f23a1d628c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:31.409296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:31.412011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5110f414-ec67-4625-b01b-f23a1d628c95 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:31.413825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5110f414-ec67-4625-b01b-f23a1d628c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:31.510778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:22:31.550121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e2f09-2bfa-4db4-85bb-fb0ed97a22d2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:31.551685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5b6e2f09-2bfa-4db4-85bb-fb0ed97a22d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:31.664688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e2f09-2bfa-4db4-85bb-fb0ed97a22d2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:31.714932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:31.739806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488268f-1392-44da-bd6c-6d6c708953be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:31.740439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488268f-1392-44da-bd6c-6d6c708953be service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:31.740717 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488268f-1392-44da-bd6c-6d6c708953be service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:31.740987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488268f-1392-44da-bd6c-6d6c708953be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:31.742591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e488268f-1392-44da-bd6c-6d6c708953be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:31.851545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488268f-1392-44da-bd6c-6d6c708953be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:31.853012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488268f-1392-44da-bd6c-6d6c708953be service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:31.871783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 731/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:33.910848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcff68-e36d-45bc-b8f8-f6e0ef1cb114 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:33.911459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcff68-e36d-45bc-b8f8-f6e0ef1cb114 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:33.911459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcff68-e36d-45bc-b8f8-f6e0ef1cb114 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:33.912772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:33.918202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800c18f-e121-4923-9cc5-7b1ce88387f7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:33.918601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800c18f-e121-4923-9cc5-7b1ce88387f7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:33.918747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800c18f-e121-4923-9cc5-7b1ce88387f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:33.985343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4800c18f-e121-4923-9cc5-7b1ce88387f7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:33.986607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4800c18f-e121-4923-9cc5-7b1ce88387f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:34.016890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:34.085381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:34.123872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcff68-e36d-45bc-b8f8-f6e0ef1cb114 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:34.125478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-57fcff68-e36d-45bc-b8f8-f6e0ef1cb114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:34.227881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcff68-e36d-45bc-b8f8-f6e0ef1cb114 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:34.256648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:34.301105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63757ceb-6417-430f-a355-8003aa517e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:34.301757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63757ceb-6417-430f-a355-8003aa517e86 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:34.302031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63757ceb-6417-430f-a355-8003aa517e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:34.302296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63757ceb-6417-430f-a355-8003aa517e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:34.304146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-63757ceb-6417-430f-a355-8003aa517e86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:34.411989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63757ceb-6417-430f-a355-8003aa517e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:34.413464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63757ceb-6417-430f-a355-8003aa517e86 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:34.431615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 726/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:36.514303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57c9c3-16b5-4c57-bf78-561bf1ac6487 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:36.514884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57c9c3-16b5-4c57-bf78-561bf1ac6487 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:36.514884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57c9c3-16b5-4c57-bf78-561bf1ac6487 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:36.516217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:36.521558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facecaa-238a-4027-bc2e-3ee78feb92fd None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:36.521968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facecaa-238a-4027-bc2e-3ee78feb92fd None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:36.522143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facecaa-238a-4027-bc2e-3ee78feb92fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:36.587970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1facecaa-238a-4027-bc2e-3ee78feb92fd None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:36.589201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1facecaa-238a-4027-bc2e-3ee78feb92fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:36.597569 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:36.684115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:36.722454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57c9c3-16b5-4c57-bf78-561bf1ac6487 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:36.723983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5e57c9c3-16b5-4c57-bf78-561bf1ac6487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:36.826994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57c9c3-16b5-4c57-bf78-561bf1ac6487 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:36.881595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:36.900194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea71e6-07ac-4962-9fc2-efceeeac3419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:36.900798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea71e6-07ac-4962-9fc2-efceeeac3419 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:36.901074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea71e6-07ac-4962-9fc2-efceeeac3419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:36.901345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea71e6-07ac-4962-9fc2-efceeeac3419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:36.902929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b3ea71e6-07ac-4962-9fc2-efceeeac3419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:37.011762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea71e6-07ac-4962-9fc2-efceeeac3419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:37.013605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea71e6-07ac-4962-9fc2-efceeeac3419 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:37.032519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 734/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:39.084634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f336f-9189-4e0c-b097-f84f90c5dc6a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:39.085100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f336f-9189-4e0c-b097-f84f90c5dc6a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:39.085217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f336f-9189-4e0c-b097-f84f90c5dc6a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:39.086550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:39.092634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99697fb0-0d9f-43cd-b242-6fec759206b6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:39.093063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99697fb0-0d9f-43cd-b242-6fec759206b6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:39.093216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99697fb0-0d9f-43cd-b242-6fec759206b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:39.159579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-99697fb0-0d9f-43cd-b242-6fec759206b6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:39.160815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99697fb0-0d9f-43cd-b242-6fec759206b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:39.201652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:39.261926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:39.300635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f336f-9189-4e0c-b097-f84f90c5dc6a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:39.302223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-932f336f-9189-4e0c-b097-f84f90c5dc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:39.405929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f336f-9189-4e0c-b097-f84f90c5dc6a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:39.442529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:39.479203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571b5f9-a5da-498c-93b1-c865cf3e048f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:39.479808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571b5f9-a5da-498c-93b1-c865cf3e048f service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:39.480084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571b5f9-a5da-498c-93b1-c865cf3e048f service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:39.480381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571b5f9-a5da-498c-93b1-c865cf3e048f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:39.482267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e571b5f9-a5da-498c-93b1-c865cf3e048f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:39.589403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571b5f9-a5da-498c-93b1-c865cf3e048f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:39.591645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571b5f9-a5da-498c-93b1-c865cf3e048f service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:39.609208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 729/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:41.690578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ac8ca-c42c-4632-a250-ca6c4eef3789 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:41.691145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ac8ca-c42c-4632-a250-ca6c4eef3789 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:41.691145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ac8ca-c42c-4632-a250-ca6c4eef3789 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:41.692552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:41.698039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d1470-ed3a-454f-b339-38e5c446d4de None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:41.698443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d1470-ed3a-454f-b339-38e5c446d4de None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:41.698605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d1470-ed3a-454f-b339-38e5c446d4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:41.764078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-591d1470-ed3a-454f-b339-38e5c446d4de None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:41.765312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-591d1470-ed3a-454f-b339-38e5c446d4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:41.783934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:41.861633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:41.898198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ac8ca-c42c-4632-a250-ca6c4eef3789 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:41.899799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bf8ac8ca-c42c-4632-a250-ca6c4eef3789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:42.005662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ac8ca-c42c-4632-a250-ca6c4eef3789 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:42.074131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:42.082740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364754ee-63d3-4bf4-9d9a-d1e7796c6c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:42.083347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364754ee-63d3-4bf4-9d9a-d1e7796c6c40 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:42.083623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364754ee-63d3-4bf4-9d9a-d1e7796c6c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:42.083894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364754ee-63d3-4bf4-9d9a-d1e7796c6c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:42.085511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-364754ee-63d3-4bf4-9d9a-d1e7796c6c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:42.194799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364754ee-63d3-4bf4-9d9a-d1e7796c6c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:42.196516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364754ee-63d3-4bf4-9d9a-d1e7796c6c40 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:42.216054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 737/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:22:44.218165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a9b9-88a6-4e22-a3f1-03e695c4927e None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:44.218697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a9b9-88a6-4e22-a3f1-03e695c4927e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:44.218697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a9b9-88a6-4e22-a3f1-03e695c4927e None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:44.220051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:22:44.225311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeedc80-46d4-4f1c-86d3-d81dcd4942d9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:44.225737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeedc80-46d4-4f1c-86d3-d81dcd4942d9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:44.225874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeedc80-46d4-4f1c-86d3-d81dcd4942d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:44.290961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1eeedc80-46d4-4f1c-86d3-d81dcd4942d9 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:22:44.292117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1eeedc80-46d4-4f1c-86d3-d81dcd4942d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:44.340783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:44.389295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:22:44.426611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5a9b9-88a6-4e22-a3f1-03e695c4927e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:22:44.428173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f4c5a9b9-88a6-4e22-a3f1-03e695c4927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:44.528625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5a9b9-88a6-4e22-a3f1-03e695c4927e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:22:44.574935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:44.600919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4faec-d1d9-45db-9dbe-d6e58b317bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:22:44.601516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4faec-d1d9-45db-9dbe-d6e58b317bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:22:44.601791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4faec-d1d9-45db-9dbe-d6e58b317bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:22:44.602056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4faec-d1d9-45db-9dbe-d6e58b317bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:22:44.603629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-37d4faec-d1d9-45db-9dbe-d6e58b317bd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:22:44.712367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4faec-d1d9-45db-9dbe-d6e58b317bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:22:44.713794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4faec-d1d9-45db-9dbe-d6e58b317bd3 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:22:44.731840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 732/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:22:54.387359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=22 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:22:54.730163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=66 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:04.242771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2d61-88e7-4b1b-b8fe-44216ac73998 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:04.243945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2d61-88e7-4b1b-b8fe-44216ac73998 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:04.243945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b2d61-88e7-4b1b-b8fe-44216ac73998 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:04.245325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 739/1471] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:23:04.250560 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa14f19-31fd-4c78-9b9b-26074c888dec None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:04.251036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa14f19-31fd-4c78-9b9b-26074c888dec None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:04.251148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa14f19-31fd-4c78-9b9b-26074c888dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:04.316599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-7aa14f19-31fd-4c78-9b9b-26074c888dec None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:23:04.317723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7aa14f19-31fd-4c78-9b9b-26074c888dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:04.411941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 733/1472] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:23:04.495802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1b0f3-3b8f-4127-9ec4-b6193ddcc5bb None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:04.496085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1b0f3-3b8f-4127-9ec4-b6193ddcc5bb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:04.496394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1b0f3-3b8f-4127-9ec4-b6193ddcc5bb None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:04.497759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 740/1473] 10.222.0.43 () {54 vars in 967 bytes} [Thu Aug 11 16:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:23:04.502370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a112a9-6def-4a56-8a7f-4213534b791d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:04.502650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a112a9-6def-4a56-8a7f-4213534b791d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:04.502918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a112a9-6def-4a56-8a7f-4213534b791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:04.576966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-17a112a9-6def-4a56-8a7f-4213534b791d None None] MFA Rules not processed for user `f7a2237c18934cb2aa3699a5776874ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:23:04.578079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-17a112a9-6def-4a56-8a7f-4213534b791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:04.598347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:04.652847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 734/1474] 10.222.0.43 () {58 vars in 1064 bytes} [Thu Aug 11 16:23:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:23:04.660653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1b0f3-3b8f-4127-9ec4-b6193ddcc5bb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:23:04.662170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-20e1b0f3-3b8f-4127-9ec4-b6193ddcc5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:04.766421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1b0f3-3b8f-4127-9ec4-b6193ddcc5bb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:23:04.837802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a86c1-9e58-444f-aa7a-876ce570d78d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7a2237c18934cb2aa3699a5776874ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:23:04.838332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a86c1-9e58-444f-aa7a-876ce570d78d service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:04.838608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a86c1-9e58-444f-aa7a-876ce570d78d service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:04.838876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a86c1-9e58-444f-aa7a-876ce570d78d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:04.840486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b59a86c1-9e58-444f-aa7a-876ce570d78d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:04.892794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:04.949259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a86c1-9e58-444f-aa7a-876ce570d78d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:23:04.950678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a86c1-9e58-444f-aa7a-876ce570d78d service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:23:04.968788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 741/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:23:07.086781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4bdb-a2f0-4c45-986a-c8134690f8d5 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:07.087465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4bdb-a2f0-4c45-986a-c8134690f8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:07.087465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c4bdb-a2f0-4c45-986a-c8134690f8d5 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:07.088713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 735/1476] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:23:07.095424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408be10-8336-4a14-aee4-15f96ad951ee None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:07.095856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408be10-8336-4a14-aee4-15f96ad951ee None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:07.095967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408be10-8336-4a14-aee4-15f96ad951ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:07.182025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b408be10-8336-4a14-aee4-15f96ad951ee None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:23:07.183269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b408be10-8336-4a14-aee4-15f96ad951ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:07.286664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 742/1477] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:23:07.310410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c4bdb-a2f0-4c45-986a-c8134690f8d5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:23:07.311879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-887c4bdb-a2f0-4c45-986a-c8134690f8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:07.444565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c4bdb-a2f0-4c45-986a-c8134690f8d5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:23:07.493829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:07.537957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a917a-773d-48fd-b687-d1d94db6feb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:23:07.538581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a917a-773d-48fd-b687-d1d94db6feb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:07.538859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a917a-773d-48fd-b687-d1d94db6feb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:07.539128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a917a-773d-48fd-b687-d1d94db6feb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:07.540412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a917a-773d-48fd-b687-d1d94db6feb8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:23:07.542235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING py.warnings [None req-bc6a917a-773d-48fd-b687-d1d94db6feb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:23:07.542235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: warnings.warn(msg) Aug 11 16:23:07.542235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: Aug 11 16:23:07.543362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a917a-773d-48fd-b687-d1d94db6feb8 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:23:07.561983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 736/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Aug 11 16:23:07 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:23:09.585988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479fdd3-6fdd-480e-832f-1f96eb6b4a56 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:09.587146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479fdd3-6fdd-480e-832f-1f96eb6b4a56 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:09.587349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479fdd3-6fdd-480e-832f-1f96eb6b4a56 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:09.588755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 743/1479] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:23:09.594106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3802032-e002-4cb8-98ad-621cd1e86af9 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:09.594501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3802032-e002-4cb8-98ad-621cd1e86af9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:09.594658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3802032-e002-4cb8-98ad-621cd1e86af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:09.659974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f3802032-e002-4cb8-98ad-621cd1e86af9 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:23:09.661379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f3802032-e002-4cb8-98ad-621cd1e86af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:09.757278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 737/1480] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:23:09.800086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479fdd3-6fdd-480e-832f-1f96eb6b4a56 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:23:09.801747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d479fdd3-6fdd-480e-832f-1f96eb6b4a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:09.812017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:09.948190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479fdd3-6fdd-480e-832f-1f96eb6b4a56 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:23:10.027050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:10.061247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481632e-065a-48c7-a4c2-4ba116173416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:23:10.061855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481632e-065a-48c7-a4c2-4ba116173416 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:10.062131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481632e-065a-48c7-a4c2-4ba116173416 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:10.062399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481632e-065a-48c7-a4c2-4ba116173416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:10.063992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1481632e-065a-48c7-a4c2-4ba116173416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:10.153757 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481632e-065a-48c7-a4c2-4ba116173416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:23:10.155351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481632e-065a-48c7-a4c2-4ba116173416 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:23:10.173700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 744/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:23:12.385513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ea1c-2628-4ad9-9f52-982f550b4202 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:12.386067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ea1c-2628-4ad9-9f52-982f550b4202 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:12.386067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ea1c-2628-4ad9-9f52-982f550b4202 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:12.387391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 738/1482] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:23:12.392499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b69cb-dbb2-48d5-99b2-16786751a78a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:12.392923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b69cb-dbb2-48d5-99b2-16786751a78a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:12.393025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b69cb-dbb2-48d5-99b2-16786751a78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:12.460511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3b7b69cb-dbb2-48d5-99b2-16786751a78a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:23:12.461782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3b7b69cb-dbb2-48d5-99b2-16786751a78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:12.507633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:12.557276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 745/1483] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:23:12.594576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9ea1c-2628-4ad9-9f52-982f550b4202 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:23:12.596153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-92f9ea1c-2628-4ad9-9f52-982f550b4202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:12.620343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:12.698574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9ea1c-2628-4ad9-9f52-982f550b4202 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:23:12.768598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31140eb8-00d0-4af1-bf54-7854499218b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:23:12.769191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31140eb8-00d0-4af1-bf54-7854499218b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:12.769464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31140eb8-00d0-4af1-bf54-7854499218b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:12.769727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31140eb8-00d0-4af1-bf54-7854499218b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:12.771305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-31140eb8-00d0-4af1-bf54-7854499218b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:12.845040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:12.880143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31140eb8-00d0-4af1-bf54-7854499218b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:23:12.881579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31140eb8-00d0-4af1-bf54-7854499218b0 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:23:12.900307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 739/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:23:14.953756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e2cb5-7ddf-492e-9c38-3ca715ecbbc9 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:14.954297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e2cb5-7ddf-492e-9c38-3ca715ecbbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:14.954297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e2cb5-7ddf-492e-9c38-3ca715ecbbc9 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:14.955654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 746/1485] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:23:14.961626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d47c-f6b3-402b-98a7-b18fc0524f1f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:14.962079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d47c-f6b3-402b-98a7-b18fc0524f1f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:14.962321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d47c-f6b3-402b-98a7-b18fc0524f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:15.029673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-7ce8d47c-f6b3-402b-98a7-b18fc0524f1f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:23:15.030885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7ce8d47c-f6b3-402b-98a7-b18fc0524f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:15.127718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 740/1486] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:23:15.166976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e2cb5-7ddf-492e-9c38-3ca715ecbbc9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:23:15.168562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-748e2cb5-7ddf-492e-9c38-3ca715ecbbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:15.276290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e2cb5-7ddf-492e-9c38-3ca715ecbbc9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:23:15.349998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:15.352604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f6b46-3393-4f44-9855-69bcfa2c0745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:23:15.353223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6b46-3393-4f44-9855-69bcfa2c0745 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:15.353700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6b46-3393-4f44-9855-69bcfa2c0745 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:15.354178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f6b46-3393-4f44-9855-69bcfa2c0745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:15.356033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-da6f6b46-3393-4f44-9855-69bcfa2c0745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:15.463509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f6b46-3393-4f44-9855-69bcfa2c0745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:23:15.464878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f6b46-3393-4f44-9855-69bcfa2c0745 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:23:15.483233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 747/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:23:17.589133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2f17-3b8f-42d5-9539-6b65c77e7e3b None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:17.589752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2f17-3b8f-42d5-9539-6b65c77e7e3b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:17.589752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce2f17-3b8f-42d5-9539-6b65c77e7e3b None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:17.591003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 741/1488] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:23:17.596384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6eae-34c0-43e8-b799-e972a39c6ba7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:17.596814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6eae-34c0-43e8-b799-e972a39c6ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:17.596979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6eae-34c0-43e8-b799-e972a39c6ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:17.664509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2e4c6eae-34c0-43e8-b799-e972a39c6ba7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:23:17.665782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e4c6eae-34c0-43e8-b799-e972a39c6ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:17.719008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:17.763476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 748/1489] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:23:17.804311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce2f17-3b8f-42d5-9539-6b65c77e7e3b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:23:17.805924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4cce2f17-3b8f-42d5-9539-6b65c77e7e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:17.843796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:17.911376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce2f17-3b8f-42d5-9539-6b65c77e7e3b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:23:17.983284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f54c69-e02f-44fc-8e19-cc2c92af6d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:23:17.983880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f54c69-e02f-44fc-8e19-cc2c92af6d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:17.984179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f54c69-e02f-44fc-8e19-cc2c92af6d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:17.984449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f54c69-e02f-44fc-8e19-cc2c92af6d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:17.986051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-86f54c69-e02f-44fc-8e19-cc2c92af6d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:18.066124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:18.095971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f54c69-e02f-44fc-8e19-cc2c92af6d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:23:18.097431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f54c69-e02f-44fc-8e19-cc2c92af6d49 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:23:18.117249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 742/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:23:20.394813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49941e4f-ad5a-46db-bb2e-c97e9512c516 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:20.395366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49941e4f-ad5a-46db-bb2e-c97e9512c516 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:20.395593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49941e4f-ad5a-46db-bb2e-c97e9512c516 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:20.397066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 749/1491] 10.222.0.43 () {54 vars in 927 bytes} [Thu Aug 11 16:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:23:20.403197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60b281-0938-421f-8840-84020bdb8824 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:20.403652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60b281-0938-421f-8840-84020bdb8824 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:20.403766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60b281-0938-421f-8840-84020bdb8824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:20.468851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5c60b281-0938-421f-8840-84020bdb8824 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:23:20.470060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5c60b281-0938-421f-8840-84020bdb8824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:20.564780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 743/1492] 10.222.0.43 () {58 vars in 1024 bytes} [Thu Aug 11 16:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:23:20.605012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49941e4f-ad5a-46db-bb2e-c97e9512c516 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:23:20.606576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-49941e4f-ad5a-46db-bb2e-c97e9512c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:20.711568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49941e4f-ad5a-46db-bb2e-c97e9512c516 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:23:20.783244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6183b6b-6452-4edb-a743-69f41776270e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:23:20.783855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6183b6b-6452-4edb-a743-69f41776270e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:23:20.784159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6183b6b-6452-4edb-a743-69f41776270e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:23:20.784435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6183b6b-6452-4edb-a743-69f41776270e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:23:20.786036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e6183b6b-6452-4edb-a743-69f41776270e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:23:20.795118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:20.894999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6183b6b-6452-4edb-a743-69f41776270e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:23:20.896363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6183b6b-6452-4edb-a743-69f41776270e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:23:20.914801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 750/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:23:30.562905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=82 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:23:30.912512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=50 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:31.213829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0acf6-20dc-4dc4-a656-8e39913a5941 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:31.214437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0acf6-20dc-4dc4-a656-8e39913a5941 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:31.214437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0acf6-20dc-4dc4-a656-8e39913a5941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:31.288405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-b3a0acf6-20dc-4dc4-a656-8e39913a5941 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:24:31.289601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b3a0acf6-20dc-4dc4-a656-8e39913a5941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:31.383419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 744/1494] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:24:31.391728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85653c79-859c-4f26-a6a5-746997d5ccc2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:31.392254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85653c79-859c-4f26-a6a5-746997d5ccc2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:31.392363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85653c79-859c-4f26-a6a5-746997d5ccc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:31.466666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-85653c79-859c-4f26-a6a5-746997d5ccc2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:24:31.467859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-85653c79-859c-4f26-a6a5-746997d5ccc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:31.562644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 751/1495] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:24:31.567538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0acf6-20dc-4dc4-a656-8e39913a5941 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:31.568791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b3a0acf6-20dc-4dc4-a656-8e39913a5941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:31.580719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0acf6-20dc-4dc4-a656-8e39913a5941 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:31.641742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:31.675137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:31.675733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:31.676005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:31.676295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:31.676930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:24:31.677223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:31.678481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING py.warnings [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:24:31.678481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: warnings.warn(msg) Aug 11 16:24:31.678481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: Aug 11 16:24:31.679351 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:31.690548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 745/1496] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:24:31.698820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85653c79-859c-4f26-a6a5-746997d5ccc2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:31.708392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85653c79-859c-4f26-a6a5-746997d5ccc2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:31.769659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:31.804165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:31.804773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:31.805052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:31.805321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:31.805955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:31.807665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:24:31.807665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 11 16:24:31.807665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: Aug 11 16:24:31.808881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:31.866081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 752/1497] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:24:31 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:24:31.870061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:31.871626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:31.907237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1174-f8b1-497b-a49c-43eabd6db53a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:31.999326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:32.000864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:32.001151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:32.001417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:32.002042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:32.003045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING py.warnings [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:24:32.003045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: warnings.warn(msg) Aug 11 16:24:32.003045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: Aug 11 16:24:32.004265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:32.050119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 746/1498] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:24:31 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:24:32.054295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:32.055907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:32.118182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25515-58f7-49c4-b260-e02b16597fed admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:32.200405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:32.213326 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:32.213929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:32.214204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:32.214471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:32.215107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:32.216316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:24:32.216316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 11 16:24:32.216316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: Aug 11 16:24:32.217192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:32.229251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 753/1499] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:24:32.233337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:32.235409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:32.246549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f5468-4aaa-40e2-a36f-907f42b3cd7a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:32.271788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:32.342551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e1205-2de8-4395-bb38-965388b0c6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:32.343164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e1205-2de8-4395-bb38-965388b0c6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:32.343614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e1205-2de8-4395-bb38-965388b0c6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:32.344012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e1205-2de8-4395-bb38-965388b0c6e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:32.344693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e1205-2de8-4395-bb38-965388b0c6e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:32.345702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING py.warnings [None req-258e1205-2de8-4395-bb38-965388b0c6e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:24:32.345702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: warnings.warn(msg) Aug 11 16:24:32.345702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: Aug 11 16:24:32.346579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e1205-2de8-4395-bb38-965388b0c6e3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:32.358718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 747/1500] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:24:32.363416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:32.364972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:32.398229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613eb4f6-ab04-479c-98b3-2905ebd615ee admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:32.491084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4356ce-b3f9-4fc2-a180-7d3a6d6b5478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:32.491660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4356ce-b3f9-4fc2-a180-7d3a6d6b5478 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:32.491934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4356ce-b3f9-4fc2-a180-7d3a6d6b5478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:32.492237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4356ce-b3f9-4fc2-a180-7d3a6d6b5478 admin admin] PATH_INFO: `/v3/projects/4b8c7571f9244240af454b4c0556c004/users/c4c1f518a78546529a4ce120d26815f9/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:32.513762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4356ce-b3f9-4fc2-a180-7d3a6d6b5478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8c7571f9244240af454b4c0556c004, user_id=c4c1f518a78546529a4ce120d26815f9, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:32.514770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-4f4356ce-b3f9-4fc2-a180-7d3a6d6b5478 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:24:32.514770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 11 16:24:32.514770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: Aug 11 16:24:32.516755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4356ce-b3f9-4fc2-a180-7d3a6d6b5478 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:32.540615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 754/1501] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:24:32 2022] PUT /identity/v3/projects/4b8c7571f9244240af454b4c0556c004/users/c4c1f518a78546529a4ce120d26815f9/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:24:32.553402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e96f-b659-49e7-a8e8-0a62e47921b7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:32.553686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e96f-b659-49e7-a8e8-0a62e47921b7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:32.553956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e96f-b659-49e7-a8e8-0a62e47921b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:32.611118 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-7cf6e96f-b659-49e7-a8e8-0a62e47921b7 None None] MFA Rules not processed for user `c4c1f518a78546529a4ce120d26815f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:24:32.612359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7cf6e96f-b659-49e7-a8e8-0a62e47921b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:32.690277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 748/1502] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:24:32.703639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd2038-6787-4f2c-94c2-2414d18a07d6 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:32.703921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd2038-6787-4f2c-94c2-2414d18a07d6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:32.704227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd2038-6787-4f2c-94c2-2414d18a07d6 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:32.705562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 755/1503] 10.222.0.43 () {54 vars in 964 bytes} [Thu Aug 11 16:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:24:32.710470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d1262-52f5-499e-9c12-d7b888648ea8 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:32.710756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d1262-52f5-499e-9c12-d7b888648ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:32.711030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d1262-52f5-499e-9c12-d7b888648ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:32.742775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:32.787733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-cd2d1262-52f5-499e-9c12-d7b888648ea8 None None] MFA Rules not processed for user `89757c5dc48942259ffdd67e0cd19ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:24:32.788973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-cd2d1262-52f5-499e-9c12-d7b888648ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:32.864157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 749/1504] 10.222.0.43 () {58 vars in 1061 bytes} [Thu Aug 11 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:24:32.872206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd2038-6787-4f2c-94c2-2414d18a07d6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:32.873708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-16dd2038-6787-4f2c-94c2-2414d18a07d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:32.924820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:32.981069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd2038-6787-4f2c-94c2-2414d18a07d6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:33.059368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:33.059977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:33.060301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:33.060576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:33.062206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:33.158503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:33.159910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:33.163321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:33.178143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 756/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:24:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:24:33.223460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120545-f848-4209-a2ba-163a8cc1acdc None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:33.223774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120545-f848-4209-a2ba-163a8cc1acdc None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:33.224046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b120545-f848-4209-a2ba-163a8cc1acdc None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:33.225347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 750/1506] 10.222.0.43 () {52 vars in 830 bytes} [Thu Aug 11 16:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:24:33.228851 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:33.230117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:33.329712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab039ddc-97c9-43b2-a4f9-345ec29688cd service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:33.421797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:33.422455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:33.422603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:33.422868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] PATH_INFO: `/v3/users/c4c1f518a78546529a4ce120d26815f9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:33.434623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c1f518a78546529a4ce120d26815f9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:33.435627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:24:33.435627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 11 16:24:33.435627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: Aug 11 16:24:33.437380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:33.447088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:33.501660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c1f518a78546529a4ce120d26815f9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:33.511699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c1f518a78546529a4ce120d26815f9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:33.512279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c1f518a78546529a4ce120d26815f9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:33.535025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c1f518a78546529a4ce120d26815f9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:33.535466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c1f518a78546529a4ce120d26815f9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:33.554325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 757/1507] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:24:33 2022] DELETE /identity/v3/users/c4c1f518a78546529a4ce120d26815f9 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:24:33.558709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f851026-685d-4313-9c21-19aa09ccbaca None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:33.559186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f851026-685d-4313-9c21-19aa09ccbaca None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:33.559287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f851026-685d-4313-9c21-19aa09ccbaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:33.616147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1f851026-685d-4313-9c21-19aa09ccbaca None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:24:33.617294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1f851026-685d-4313-9c21-19aa09ccbaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:33.696280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:33.709547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 751/1508] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:24:33.719091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:33.720425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:33.822481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e3b69-fe51-453f-8945-bc0247df163a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:33.879581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:33.901285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78536530-c1a9-44d0-bffd-8b838d39bda1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:33.901945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536530-c1a9-44d0-bffd-8b838d39bda1 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:33.902219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536530-c1a9-44d0-bffd-8b838d39bda1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:33.902485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78536530-c1a9-44d0-bffd-8b838d39bda1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:33.904125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-78536530-c1a9-44d0-bffd-8b838d39bda1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:34.009187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536530-c1a9-44d0-bffd-8b838d39bda1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:34.010606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78536530-c1a9-44d0-bffd-8b838d39bda1 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:34.029590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 758/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:24:34.448941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f851026-685d-4313-9c21-19aa09ccbaca None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:24:34.450226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1f851026-685d-4313-9c21-19aa09ccbaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:24:34.462179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f851026-685d-4313-9c21-19aa09ccbaca None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:24:34.564863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:24:34.565529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:24:34.565754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:24:34.566021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] PATH_INFO: `/v3/projects/4b8c7571f9244240af454b4c0556c004` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:24:34.575634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b8c7571f9244240af454b4c0556c004)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:24:34.576685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING py.warnings [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:24:34.576685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: warnings.warn(msg) Aug 11 16:24:34.576685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: Aug 11 16:24:34.577937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:24:34.635467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8c7571f9244240af454b4c0556c004', 'request_id': 'req-3a45a920-21c6-456c-a564-4a87eb1a48c3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:34.649087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8c7571f9244240af454b4c0556c004', 'request_id': 'req-3a45a920-21c6-456c-a564-4a87eb1a48c3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:34.655085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8c7571f9244240af454b4c0556c004', 'request_id': 'req-3a45a920-21c6-456c-a564-4a87eb1a48c3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:34.682274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] The token cache is being invalidate because project 4b8c7571f9244240af454b4c0556c004 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:24:34.682560 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:34.683076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3a45a920-21c6-456c-a564-4a87eb1a48c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:24:34.684424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 752/1510] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:24:34 2022] DELETE /identity/v3/projects/4b8c7571f9244240af454b4c0556c004 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:24:44.027330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=62 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:24:44.680300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:26:02.166014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15092f-781b-40b2-aa25-f4714a2307a5 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:02.166611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15092f-781b-40b2-aa25-f4714a2307a5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:02.166611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15092f-781b-40b2-aa25-f4714a2307a5 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:02.167931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:26:02.173287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b4c2c-af68-4ed5-b0fd-21b17fe8fc7f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:02.173708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b4c2c-af68-4ed5-b0fd-21b17fe8fc7f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:02.173863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b4c2c-af68-4ed5-b0fd-21b17fe8fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:02.259272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-665b4c2c-af68-4ed5-b0fd-21b17fe8fc7f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:02.260858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-665b4c2c-af68-4ed5-b0fd-21b17fe8fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:02.366657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:02.537526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15092f-781b-40b2-aa25-f4714a2307a5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:02.539640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fd15092f-781b-40b2-aa25-f4714a2307a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:02.660442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15092f-781b-40b2-aa25-f4714a2307a5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:02.665804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0526a-91ba-492d-9215-f35025bd9189 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:02.666407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0526a-91ba-492d-9215-f35025bd9189 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:02.666753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0526a-91ba-492d-9215-f35025bd9189 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:02.666967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0526a-91ba-492d-9215-f35025bd9189 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:02.675192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0526a-91ba-492d-9215-f35025bd9189 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:02.676812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0526a-91ba-492d-9215-f35025bd9189 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:02.687001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 760/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:26:03.536051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e101b-26f7-45c0-a3f3-99111b2e89be None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:03.536614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e101b-26f7-45c0-a3f3-99111b2e89be None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:03.536614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e101b-26f7-45c0-a3f3-99111b2e89be None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:03.537972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:26:03.543873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546134eb-7762-438d-97d8-9a6aa812fee1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:03.544353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546134eb-7762-438d-97d8-9a6aa812fee1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:03.544463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546134eb-7762-438d-97d8-9a6aa812fee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:03.563814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-546134eb-7762-438d-97d8-9a6aa812fee1 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:03.565065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-546134eb-7762-438d-97d8-9a6aa812fee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:03.573036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:03.745973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e101b-26f7-45c0-a3f3-99111b2e89be None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:03.760899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e101b-26f7-45c0-a3f3-99111b2e89be None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:03.766854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d232c-43a1-4b75-84f7-d9bc60cf5dbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:03.767438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d232c-43a1-4b75-84f7-d9bc60cf5dbe service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:03.767721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d232c-43a1-4b75-84f7-d9bc60cf5dbe service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:03.767988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d232c-43a1-4b75-84f7-d9bc60cf5dbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:03.776445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d232c-43a1-4b75-84f7-d9bc60cf5dbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:03.778244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d232c-43a1-4b75-84f7-d9bc60cf5dbe service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:03.789386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 755/1516] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:26:04.614953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6d788-61af-4b68-9540-5b7bf1f011ba None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:04.615476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6d788-61af-4b68-9540-5b7bf1f011ba None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:04.615635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6d788-61af-4b68-9540-5b7bf1f011ba None None] PATH_INFO: `/v3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:04.617010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 762/1517] 10.222.0.16 () {54 vars in 925 bytes} [Thu Aug 11 16:26:04 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:26:04.624826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992525-92c1-4ef5-b9bf-5d921701c323 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:04.625335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992525-92c1-4ef5-b9bf-5d921701c323 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:04.625431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c992525-92c1-4ef5-b9bf-5d921701c323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:04.689235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-6c992525-92c1-4ef5-b9bf-5d921701c323 None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:04.690481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6c992525-92c1-4ef5-b9bf-5d921701c323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:04.710457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:26:04.769563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 756/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Aug 11 16:26:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:08.573705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74f07-7076-4e91-b696-135abeddff79 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:08.574268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74f07-7076-4e91-b696-135abeddff79 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:08.574268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f74f07-7076-4e91-b696-135abeddff79 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:08.575600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 763/1519] 10.222.0.43 () {54 vars in 961 bytes} [Thu Aug 11 16:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:26:08.580504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a11024-d709-4d01-90ab-8d68b7bc15d0 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:08.580903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a11024-d709-4d01-90ab-8d68b7bc15d0 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:08.581057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a11024-d709-4d01-90ab-8d68b7bc15d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:08.600582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d6a11024-d709-4d01-90ab-8d68b7bc15d0 None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:08.601785 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d6a11024-d709-4d01-90ab-8d68b7bc15d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:08.611034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 757/1520] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Aug 11 16:26:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:26:08.618438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f74f07-7076-4e91-b696-135abeddff79 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:08.631810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f74f07-7076-4e91-b696-135abeddff79 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:08.636874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc13422-39d9-428c-a503-3cd6f5da055a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:08.637488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc13422-39d9-428c-a503-3cd6f5da055a service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:08.637766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc13422-39d9-428c-a503-3cd6f5da055a service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:08.638037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc13422-39d9-428c-a503-3cd6f5da055a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:08.646034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc13422-39d9-428c-a503-3cd6f5da055a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:08.647628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc13422-39d9-428c-a503-3cd6f5da055a service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:08.657566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 764/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:26:15.061810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf3a44-fd23-4ebf-b05a-3fbb58c99c35 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:15.062302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf3a44-fd23-4ebf-b05a-3fbb58c99c35 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:15.062413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf3a44-fd23-4ebf-b05a-3fbb58c99c35 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:15.063674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:26:15.068553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceef15-a07c-4a58-b341-4f93066ac350 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:15.069002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceef15-a07c-4a58-b341-4f93066ac350 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:15.069085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceef15-a07c-4a58-b341-4f93066ac350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:15.090501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2eceef15-a07c-4a58-b341-4f93066ac350 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:15.091718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2eceef15-a07c-4a58-b341-4f93066ac350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:15.099430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:15.255843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf3a44-fd23-4ebf-b05a-3fbb58c99c35 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:15.271473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf3a44-fd23-4ebf-b05a-3fbb58c99c35 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:15.277063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d545f-df28-4da0-be24-7444ba525a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:15.277663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d545f-df28-4da0-be24-7444ba525a41 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:15.277940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d545f-df28-4da0-be24-7444ba525a41 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:15.278215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d545f-df28-4da0-be24-7444ba525a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:15.286540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d545f-df28-4da0-be24-7444ba525a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:15.288136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d545f-df28-4da0-be24-7444ba525a41 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:15.297443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 759/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:26:16.123340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80874cd2-d767-4599-8f03-59e31b824b95 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:16.123807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80874cd2-d767-4599-8f03-59e31b824b95 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:16.123917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80874cd2-d767-4599-8f03-59e31b824b95 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:16.125276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 11 16:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:26:16.130027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b83c7-a67b-4551-90b2-4a96f0f5e92a None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:16.130436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b83c7-a67b-4551-90b2-4a96f0f5e92a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:16.130578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b83c7-a67b-4551-90b2-4a96f0f5e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:16.150039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d59b83c7-a67b-4551-90b2-4a96f0f5e92a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:16.151278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d59b83c7-a67b-4551-90b2-4a96f0f5e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:16.159215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 11 16:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:16.316418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80874cd2-d767-4599-8f03-59e31b824b95 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:16.332981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80874cd2-d767-4599-8f03-59e31b824b95 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:16.338560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6929fc-5b94-45bd-81f7-d3f9413547d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:16.339168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6929fc-5b94-45bd-81f7-d3f9413547d9 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:16.339449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6929fc-5b94-45bd-81f7-d3f9413547d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:16.339874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6929fc-5b94-45bd-81f7-d3f9413547d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:16.348116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6929fc-5b94-45bd-81f7-d3f9413547d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:16.349706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6929fc-5b94-45bd-81f7-d3f9413547d9 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:16.359729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 767/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:26:20.243594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ff995-d22b-4441-87b6-4e921257d8d8 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:20.244328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ff995-d22b-4441-87b6-4e921257d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:20.244328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ff995-d22b-4441-87b6-4e921257d8d8 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:20.245621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:26:20.251157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b0392-6202-4f55-941a-c2a1181fb349 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:20.251735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b0392-6202-4f55-941a-c2a1181fb349 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:20.251735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b0392-6202-4f55-941a-c2a1181fb349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:20.271570 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9b9b0392-6202-4f55-941a-c2a1181fb349 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:20.272801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9b9b0392-6202-4f55-941a-c2a1181fb349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:20.281380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:20.362337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ff995-d22b-4441-87b6-4e921257d8d8 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:20.364069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-945ff995-d22b-4441-87b6-4e921257d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:20.442087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ff995-d22b-4441-87b6-4e921257d8d8 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:20.445695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82605d-29bc-476f-af51-969b0068340f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:20.446292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82605d-29bc-476f-af51-969b0068340f service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:20.446571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82605d-29bc-476f-af51-969b0068340f service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:20.446844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82605d-29bc-476f-af51-969b0068340f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:20.456236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82605d-29bc-476f-af51-969b0068340f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:20.457661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82605d-29bc-476f-af51-969b0068340f service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:20.467608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 762/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:26:22.469295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b693c-1759-4ad7-9a5a-12e15d7b8f02 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:22.469858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b693c-1759-4ad7-9a5a-12e15d7b8f02 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:22.469858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b693c-1759-4ad7-9a5a-12e15d7b8f02 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:22.471118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:26:22.476244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea264cb1-3286-4820-8de8-2b00ce0fe1d7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:22.476663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea264cb1-3286-4820-8de8-2b00ce0fe1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:22.476791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea264cb1-3286-4820-8de8-2b00ce0fe1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:22.496914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ea264cb1-3286-4820-8de8-2b00ce0fe1d7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:22.498098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ea264cb1-3286-4820-8de8-2b00ce0fe1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:22.505744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:22.543609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b693c-1759-4ad7-9a5a-12e15d7b8f02 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:22.554240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b693c-1759-4ad7-9a5a-12e15d7b8f02 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:22.557770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3276d-d411-4e6c-b866-938c9a858a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:22.558625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3276d-d411-4e6c-b866-938c9a858a98 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:22.558908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3276d-d411-4e6c-b866-938c9a858a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:22.559163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3276d-d411-4e6c-b866-938c9a858a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:22.567798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3276d-d411-4e6c-b866-938c9a858a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:22.569238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3276d-d411-4e6c-b866-938c9a858a98 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:22.578990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 770/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:26:26.093792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e0e7b-bbaf-4dae-b1fc-87213abf8cc3 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:26.094259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e0e7b-bbaf-4dae-b1fc-87213abf8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:26.094368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e0e7b-bbaf-4dae-b1fc-87213abf8cc3 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:26.095672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:26:26.100904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9872b0-01cb-4dec-b463-a8d15b3079fc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:26.101344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9872b0-01cb-4dec-b463-a8d15b3079fc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:26.101450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9872b0-01cb-4dec-b463-a8d15b3079fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:26.109777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=81 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:26:26.122949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7e9872b0-01cb-4dec-b463-a8d15b3079fc None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:26.124165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7e9872b0-01cb-4dec-b463-a8d15b3079fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:26.131866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:26.433033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e0e7b-bbaf-4dae-b1fc-87213abf8cc3 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:26.435008 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d40e0e7b-bbaf-4dae-b1fc-87213abf8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:26.498108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:26:26.512069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e0e7b-bbaf-4dae-b1fc-87213abf8cc3 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:26.515606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e750dba-958f-40a2-a6b5-ce2ec6023c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:26.516227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750dba-958f-40a2-a6b5-ce2ec6023c23 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:26.516504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750dba-958f-40a2-a6b5-ce2ec6023c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:26.516768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750dba-958f-40a2-a6b5-ce2ec6023c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:26.525373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e750dba-958f-40a2-a6b5-ce2ec6023c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:26.527492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e750dba-958f-40a2-a6b5-ce2ec6023c23 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:26.536652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 765/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:26:28.536421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082f74d-57a9-41aa-a8a1-a27af328658a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:28.536951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082f74d-57a9-41aa-a8a1-a27af328658a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:28.536951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082f74d-57a9-41aa-a8a1-a27af328658a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:28.538220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 11 16:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:26:28.543395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b98d4-8ca7-4e59-a5c4-b6598f310eea None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:28.543801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b98d4-8ca7-4e59-a5c4-b6598f310eea None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:28.543943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b98d4-8ca7-4e59-a5c4-b6598f310eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:28.563506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-a89b98d4-8ca7-4e59-a5c4-b6598f310eea None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:26:28.564734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a89b98d4-8ca7-4e59-a5c4-b6598f310eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:28.572597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 11 16:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:26:28.900511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082f74d-57a9-41aa-a8a1-a27af328658a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:26:28.902335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f082f74d-57a9-41aa-a8a1-a27af328658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:26:28.915166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082f74d-57a9-41aa-a8a1-a27af328658a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:26:28.918677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014f61-be5b-45c5-b728-e576466a8d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:26:28.919264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014f61-be5b-45c5-b728-e576466a8d7f service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:26:28.919535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014f61-be5b-45c5-b728-e576466a8d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:26:28.919799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014f61-be5b-45c5-b728-e576466a8d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:26:28.928451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf014f61-be5b-45c5-b728-e576466a8d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:26:28.929860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf014f61-be5b-45c5-b728-e576466a8d7f service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:26:28.939793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 773/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:26:38.558516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=11 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:26:38.935696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=8 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:51.929848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167e33-5526-47f6-b9a1-d2a82b2cacbf None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:51.931502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167e33-5526-47f6-b9a1-d2a82b2cacbf None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:51.932273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6167e33-5526-47f6-b9a1-d2a82b2cacbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:51.969376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-a6167e33-5526-47f6-b9a1-d2a82b2cacbf None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:51.970979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a6167e33-5526-47f6-b9a1-d2a82b2cacbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:51.978508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706b22-929b-4edf-8c0b-f6888bcfba54 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:51.979529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706b22-929b-4edf-8c0b-f6888bcfba54 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:51.980015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706b22-929b-4edf-8c0b-f6888bcfba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:51.986370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 767/1540] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:27:51.996555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4352ea4-8887-42e2-a855-c177a6f5bd58 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:51.996887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4352ea4-8887-42e2-a855-c177a6f5bd58 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:51.997102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4352ea4-8887-42e2-a855-c177a6f5bd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.006463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-32706b22-929b-4edf-8c0b-f6888bcfba54 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.007683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-32706b22-929b-4edf-8c0b-f6888bcfba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.027231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 774/1541] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:27:52.029432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-a4352ea4-8887-42e2-a855-c177a6f5bd58 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.030706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a4352ea4-8887-42e2-a855-c177a6f5bd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.036564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be285a2-3a46-4654-99aa-1c92f465bb72 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.036932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be285a2-3a46-4654-99aa-1c92f465bb72 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.037110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be285a2-3a46-4654-99aa-1c92f465bb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.047170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 768/1542] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:27:52.052259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4352ea4-8887-42e2-a855-c177a6f5bd58 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.068242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4352ea4-8887-42e2-a855-c177a6f5bd58 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.078482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2be285a2-3a46-4654-99aa-1c92f465bb72 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.079739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2be285a2-3a46-4654-99aa-1c92f465bb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.088942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.089792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.090078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.090355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.091046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:52.091352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.093407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.095755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 775/1543] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:27:52.100857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be285a2-3a46-4654-99aa-1c92f465bb72 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.117268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be285a2-3a46-4654-99aa-1c92f465bb72 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.122555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 769/1544] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:27:52.128473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.129105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.129383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.129650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.130308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:52.130607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.131075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.131769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:27:52.131769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 11 16:27:52.131769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: Aug 11 16:27:52.132651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.155251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486397-0f5c-4f71-b5b7-021e824c529e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.161111 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 776/1545] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:27:52.163495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.164403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.165654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.166254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.167133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.168712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING py.warnings [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:27:52.168712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: warnings.warn(msg) Aug 11 16:27:52.168712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: Aug 11 16:27:52.169798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.171225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.186089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22250f93-697c-4e15-9b43-f9cf70a10177 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.196767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db20182-fb92-40b6-9dae-d1dfd671ebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.197686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db20182-fb92-40b6-9dae-d1dfd671ebcc admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.198147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db20182-fb92-40b6-9dae-d1dfd671ebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.198580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db20182-fb92-40b6-9dae-d1dfd671ebcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.199389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db20182-fb92-40b6-9dae-d1dfd671ebcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.201480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db20182-fb92-40b6-9dae-d1dfd671ebcc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.262642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 777/1546] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:27:52.264072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db20182-fb92-40b6-9dae-d1dfd671ebcc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.264455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 770/1547] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:27:52.268519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.390430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db20182-fb92-40b6-9dae-d1dfd671ebcc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.408520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3fe7c-5e8a-4136-b66f-66d084cf1941 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.429364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593c2f6-38d3-4ea0-94d4-100d4ec76874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.431526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593c2f6-38d3-4ea0-94d4-100d4ec76874 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.432257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593c2f6-38d3-4ea0-94d4-100d4ec76874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.432702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593c2f6-38d3-4ea0-94d4-100d4ec76874 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.433910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593c2f6-38d3-4ea0-94d4-100d4ec76874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.436017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593c2f6-38d3-4ea0-94d4-100d4ec76874 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.472404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 771/1548] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:52.473664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e74d0a-6e1a-497f-baae-0904ab275907 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.473951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e74d0a-6e1a-497f-baae-0904ab275907 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.474228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e74d0a-6e1a-497f-baae-0904ab275907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.479217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad767d5-a897-48ec-84c9-3ae6c4dcb286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.479844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad767d5-a897-48ec-84c9-3ae6c4dcb286 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.480157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad767d5-a897-48ec-84c9-3ae6c4dcb286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.480488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad767d5-a897-48ec-84c9-3ae6c4dcb286 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.481053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad767d5-a897-48ec-84c9-3ae6c4dcb286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.482057 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-9ad767d5-a897-48ec-84c9-3ae6c4dcb286 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:27:52.482057 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 11 16:27:52.482057 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: Aug 11 16:27:52.483262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad767d5-a897-48ec-84c9-3ae6c4dcb286 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.499884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-30e74d0a-6e1a-497f-baae-0904ab275907 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.501293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-30e74d0a-6e1a-497f-baae-0904ab275907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.511667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 772/1549] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.514068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd30df-fe5d-45c8-a183-75e490058be7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.514068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd30df-fe5d-45c8-a183-75e490058be7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.514068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd30df-fe5d-45c8-a183-75e490058be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.517758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 778/1550] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:52.519162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86988922-e4a5-4939-88e4-460a47c16c4f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.519628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86988922-e4a5-4939-88e4-460a47c16c4f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.519907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86988922-e4a5-4939-88e4-460a47c16c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.534467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4ddd30df-fe5d-45c8-a183-75e490058be7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.535657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4ddd30df-fe5d-45c8-a183-75e490058be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.539460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-86988922-e4a5-4939-88e4-460a47c16c4f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.540985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-86988922-e4a5-4939-88e4-460a47c16c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.543703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 773/1551] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.549251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 779/1552] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.549539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2791b-91b2-47fd-bc37-42f9ce1852f4 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.550025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2791b-91b2-47fd-bc37-42f9ce1852f4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.550477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4821-8b0c-43dd-b25a-5c0fda87873c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.550616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2791b-91b2-47fd-bc37-42f9ce1852f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.551082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4821-8b0c-43dd-b25a-5c0fda87873c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.551394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b4821-8b0c-43dd-b25a-5c0fda87873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.571793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d90b4821-8b0c-43dd-b25a-5c0fda87873c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.572469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5dd2791b-91b2-47fd-bc37-42f9ce1852f4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.573021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d90b4821-8b0c-43dd-b25a-5c0fda87873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.573908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5dd2791b-91b2-47fd-bc37-42f9ce1852f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.581002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 780/1553] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.581728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 774/1554] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.582700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac2e12-40d1-4847-a3b3-3969b547ce16 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.582974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac2e12-40d1-4847-a3b3-3969b547ce16 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.583247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac2e12-40d1-4847-a3b3-3969b547ce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.588423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b4821-8b0c-43dd-b25a-5c0fda87873c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.598361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b4821-8b0c-43dd-b25a-5c0fda87873c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.602698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-25ac2e12-40d1-4847-a3b3-3969b547ce16 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.603329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bc14f-a1ad-4a4b-a8a6-06a6b78f7cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.603886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-25ac2e12-40d1-4847-a3b3-3969b547ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.603999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bc14f-a1ad-4a4b-a8a6-06a6b78f7cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.604245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bc14f-a1ad-4a4b-a8a6-06a6b78f7cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.604523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bc14f-a1ad-4a4b-a8a6-06a6b78f7cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.605164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bc14f-a1ad-4a4b-a8a6-06a6b78f7cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.606758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bc14f-a1ad-4a4b-a8a6-06a6b78f7cc0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.611692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 775/1555] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.612934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d3042-082b-4d6e-8be8-61ff1d7d1268 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.613217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d3042-082b-4d6e-8be8-61ff1d7d1268 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.613486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d3042-082b-4d6e-8be8-61ff1d7d1268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.622858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 781/1556] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:52.623897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bc14f-a1ad-4a4b-a8a6-06a6b78f7cc0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.634036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bc14f-a1ad-4a4b-a8a6-06a6b78f7cc0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.636029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-254d3042-082b-4d6e-8be8-61ff1d7d1268 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.637244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-254d3042-082b-4d6e-8be8-61ff1d7d1268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.638694 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3325d-c089-4680-8710-1f89195e4a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.639284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3325d-c089-4680-8710-1f89195e4a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.639579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3325d-c089-4680-8710-1f89195e4a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.639841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3325d-c089-4680-8710-1f89195e4a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.640646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3325d-c089-4680-8710-1f89195e4a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.642150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3325d-c089-4680-8710-1f89195e4a25 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.645519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 776/1557] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.646469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae475ba-0471-4b22-8715-85ca17721ca8 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.646747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae475ba-0471-4b22-8715-85ca17721ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.647604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae475ba-0471-4b22-8715-85ca17721ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.652434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 782/1558] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:52.653799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fef62-5c4a-4d7f-a1fa-4772c6f919e2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.654085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fef62-5c4a-4d7f-a1fa-4772c6f919e2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.654390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fef62-5c4a-4d7f-a1fa-4772c6f919e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.667841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-fae475ba-0471-4b22-8715-85ca17721ca8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.669115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fae475ba-0471-4b22-8715-85ca17721ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.672678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1f5fef62-5c4a-4d7f-a1fa-4772c6f919e2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.673832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1f5fef62-5c4a-4d7f-a1fa-4772c6f919e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.677193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 777/1559] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.678356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5102-a72f-4062-86db-9e46f3f44c22 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.678635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5102-a72f-4062-86db-9e46f3f44c22 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.678905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5102-a72f-4062-86db-9e46f3f44c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.688175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 783/1560] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.689668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132d0d-cf81-4abd-ad49-962fbd018d4f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.690029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132d0d-cf81-4abd-ad49-962fbd018d4f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.690368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26132d0d-cf81-4abd-ad49-962fbd018d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.705632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-176e5102-a72f-4062-86db-9e46f3f44c22 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.706827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-176e5102-a72f-4062-86db-9e46f3f44c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.708162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:52.714245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-26132d0d-cf81-4abd-ad49-962fbd018d4f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.715394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 778/1561] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.716014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-26132d0d-cf81-4abd-ad49-962fbd018d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.716729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cb69f-231c-43fa-9a26-4fc43ff3f058 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.717062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cb69f-231c-43fa-9a26-4fc43ff3f058 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.717332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cb69f-231c-43fa-9a26-4fc43ff3f058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.726415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 784/1562] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.727788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26132d0d-cf81-4abd-ad49-962fbd018d4f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.737893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26132d0d-cf81-4abd-ad49-962fbd018d4f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.738219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-470cb69f-231c-43fa-9a26-4fc43ff3f058 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:52.739412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-470cb69f-231c-43fa-9a26-4fc43ff3f058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:52.742647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba2e6-b295-4f56-abc6-306e86d82258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.744126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba2e6-b295-4f56-abc6-306e86d82258 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.744408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba2e6-b295-4f56-abc6-306e86d82258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.744675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba2e6-b295-4f56-abc6-306e86d82258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.745310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeba2e6-b295-4f56-abc6-306e86d82258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.746920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeba2e6-b295-4f56-abc6-306e86d82258 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.748564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 779/1563] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:52.749369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cb69f-231c-43fa-9a26-4fc43ff3f058 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.757444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 785/1564] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:52.758498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba2e6-b295-4f56-abc6-306e86d82258 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.763627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cb69f-231c-43fa-9a26-4fc43ff3f058 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.767936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba2e6-b295-4f56-abc6-306e86d82258 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.768363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.768962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.769241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.769512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.770220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:52.770525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.772130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.772437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc23e0-cc60-4fea-a923-e122dcb078c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.773110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc23e0-cc60-4fea-a923-e122dcb078c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.773386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc23e0-cc60-4fea-a923-e122dcb078c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.773658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc23e0-cc60-4fea-a923-e122dcb078c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.774373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc23e0-cc60-4fea-a923-e122dcb078c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.775950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc23e0-cc60-4fea-a923-e122dcb078c1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.783858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 780/1565] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:27:52.784673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.786268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 786/1566] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:52.787046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc23e0-cc60-4fea-a923-e122dcb078c1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.797520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc23e0-cc60-4fea-a923-e122dcb078c1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.799786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad87eec-2cfa-4c3b-80a3-09994b11fa62 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.802499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.803083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.803359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.803628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.804300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:52.804434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.804600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.804988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.805324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.806149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.807046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:52.807163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.807342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.808915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.819616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 787/1567] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:27:52.826798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 781/1568] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:27:52.826798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.827622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.830934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e43b6c-4f3a-4773-b748-0ec78da6bbb5 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.835477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.836053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.836355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.836650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.837131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7de2e4-b686-43ca-aad9-f6a71e2d00ff admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.837266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:52.837566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.839116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.841625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.842216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.842546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.842819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.843452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:52.843750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.845343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.847948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:52.851903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 782/1569] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:27:52.854321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.858165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 788/1570] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:27:52.859086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.866190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e527-8705-4d9c-91f2-df8204cf5d2a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.869802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bf5e4-86ae-47f7-849e-aa8ecd353d34 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.871709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.872350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.872631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.872954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.873607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:52.873910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.874459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066e5d3-ca5f-49ed-b1fb-a7a2476270f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.875444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066e5d3-ca5f-49ed-b1fb-a7a2476270f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.875576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.875721 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066e5d3-ca5f-49ed-b1fb-a7a2476270f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.875990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066e5d3-ca5f-49ed-b1fb-a7a2476270f0 admin admin] PATH_INFO: `/v3/projects/6d15983e6d8e4ad6887105f88f187044/users/3a5ab962b58f473f874bd94bfe2db6b3/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.888034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 783/1571] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:27:52.888700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.898053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d8086-1b96-436e-b4a5-a8c3a4a8c454 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:52.898192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066e5d3-ca5f-49ed-b1fb-a7a2476270f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d15983e6d8e4ad6887105f88f187044, user_id=3a5ab962b58f473f874bd94bfe2db6b3, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.900821 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066e5d3-ca5f-49ed-b1fb-a7a2476270f0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.903174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:52.903813 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:52.904112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:52.904392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] PATH_INFO: `/v3/projects/82d7c413ee4c438ab8223b42a3a5f692/users/aa6eadb8625649e29ef26b437fe81a39/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:52.915168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 789/1572] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:52 2022] PUT /identity/v3/projects/6d15983e6d8e4ad6887105f88f187044/users/3a5ab962b58f473f874bd94bfe2db6b3/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:52.916788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066e5d3-ca5f-49ed-b1fb-a7a2476270f0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.923188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d7c413ee4c438ab8223b42a3a5f692, user_id=aa6eadb8625649e29ef26b437fe81a39, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:52.924291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING py.warnings [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:27:52.924291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: warnings.warn(msg) Aug 11 16:27:52.924291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: Aug 11 16:27:52.926224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:52.940371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 784/1573] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:52 2022] PUT /identity/v3/projects/82d7c413ee4c438ab8223b42a3a5f692/users/aa6eadb8625649e29ef26b437fe81a39/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:52.941217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:52.988239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066e5d3-ca5f-49ed-b1fb-a7a2476270f0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.002483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c441a8-1cb6-4ae5-b2df-9016c5f37f0e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.007114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871001a8-6a33-446c-a577-7f52e35d3b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.007706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871001a8-6a33-446c-a577-7f52e35d3b0f admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.007982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871001a8-6a33-446c-a577-7f52e35d3b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.008311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871001a8-6a33-446c-a577-7f52e35d3b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.008939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871001a8-6a33-446c-a577-7f52e35d3b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.010920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871001a8-6a33-446c-a577-7f52e35d3b0f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.040848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 785/1574] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.042179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871001a8-6a33-446c-a577-7f52e35d3b0f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.051328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64228609-db85-4a5a-9c50-3885742185e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.052787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228609-db85-4a5a-9c50-3885742185e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.053069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228609-db85-4a5a-9c50-3885742185e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.053348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228609-db85-4a5a-9c50-3885742185e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.053978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64228609-db85-4a5a-9c50-3885742185e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.056859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64228609-db85-4a5a-9c50-3885742185e0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.087229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 790/1575] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:52 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.087229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64228609-db85-4a5a-9c50-3885742185e0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.105640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:53.106575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871001a8-6a33-446c-a577-7f52e35d3b0f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.150644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64228609-db85-4a5a-9c50-3885742185e0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.155301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032230c-29b5-43ab-9289-dce1eab0ebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.155915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032230c-29b5-43ab-9289-dce1eab0ebb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.156228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032230c-29b5-43ab-9289-dce1eab0ebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.156506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032230c-29b5-43ab-9289-dce1eab0ebb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.156660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:53.157131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032230c-29b5-43ab-9289-dce1eab0ebb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.159041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032230c-29b5-43ab-9289-dce1eab0ebb1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.164502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cdc7f-e5d7-4461-98bc-ce48527bd3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.165865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cdc7f-e5d7-4461-98bc-ce48527bd3fe admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.166144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cdc7f-e5d7-4461-98bc-ce48527bd3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.166414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cdc7f-e5d7-4461-98bc-ce48527bd3fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.167039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cdc7f-e5d7-4461-98bc-ce48527bd3fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.168953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cdc7f-e5d7-4461-98bc-ce48527bd3fe admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.187476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 791/1576] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.188433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032230c-29b5-43ab-9289-dce1eab0ebb1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.199545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 786/1577] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.200635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cdc7f-e5d7-4461-98bc-ce48527bd3fe admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.253109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032230c-29b5-43ab-9289-dce1eab0ebb1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.257898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b2600-3b1a-41b5-97d9-c1c52243cc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.258542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b2600-3b1a-41b5-97d9-c1c52243cc73 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.258819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b2600-3b1a-41b5-97d9-c1c52243cc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.259091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b2600-3b1a-41b5-97d9-c1c52243cc73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.259713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b2600-3b1a-41b5-97d9-c1c52243cc73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.260863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cdc7f-e5d7-4461-98bc-ce48527bd3fe admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.261685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b2600-3b1a-41b5-97d9-c1c52243cc73 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.265683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b71da78-3b54-4538-b290-b36dded83ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.266278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71da78-3b54-4538-b290-b36dded83ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.266559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71da78-3b54-4538-b290-b36dded83ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.266836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b71da78-3b54-4538-b290-b36dded83ef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.267456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71da78-3b54-4538-b290-b36dded83ef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.269375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b71da78-3b54-4538-b290-b36dded83ef2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.289613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 792/1578] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.294679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c1be8-b160-43aa-b304-d29253ed1a3f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.294966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c1be8-b160-43aa-b304-d29253ed1a3f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.295236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c1be8-b160-43aa-b304-d29253ed1a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.297727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 787/1579] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.298966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c2b6-cef1-4f31-a220-349522567309 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.299255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c2b6-cef1-4f31-a220-349522567309 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.299530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223c2b6-cef1-4f31-a220-349522567309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.315434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fb9c1be8-b160-43aa-b304-d29253ed1a3f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:53.316688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fb9c1be8-b160-43aa-b304-d29253ed1a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:53.327596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f223c2b6-cef1-4f31-a220-349522567309 None None] MFA Rules not processed for user `aa6eadb8625649e29ef26b437fe81a39`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:53.328837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f223c2b6-cef1-4f31-a220-349522567309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:53.374825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 793/1580] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:53.375724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c1be8-b160-43aa-b304-d29253ed1a3f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.385785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c1be8-b160-43aa-b304-d29253ed1a3f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.390414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607c7c3-ea8b-4cef-945f-5c73dafeb3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.391015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607c7c3-ea8b-4cef-945f-5c73dafeb3db admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.391290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607c7c3-ea8b-4cef-945f-5c73dafeb3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.391556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607c7c3-ea8b-4cef-945f-5c73dafeb3db admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.392197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607c7c3-ea8b-4cef-945f-5c73dafeb3db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.394081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607c7c3-ea8b-4cef-945f-5c73dafeb3db admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.396878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 788/1581] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:53.397812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223c2b6-cef1-4f31-a220-349522567309 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.408602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223c2b6-cef1-4f31-a220-349522567309 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.413252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9ab5e-f772-4779-b61c-6c232e264bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.413893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ab5e-f772-4779-b61c-6c232e264bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.414191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ab5e-f772-4779-b61c-6c232e264bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.414566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ab5e-f772-4779-b61c-6c232e264bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.415105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9ab5e-f772-4779-b61c-6c232e264bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.417049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9ab5e-f772-4779-b61c-6c232e264bc9 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.419320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 794/1582] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/users => generated 291 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.420155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607c7c3-ea8b-4cef-945f-5c73dafeb3db admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.430819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607c7c3-ea8b-4cef-945f-5c73dafeb3db admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.436497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ea8d5-d3fa-460e-ac52-38bec4107ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.437115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea8d5-d3fa-460e-ac52-38bec4107ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.437394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea8d5-d3fa-460e-ac52-38bec4107ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.437668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea8d5-d3fa-460e-ac52-38bec4107ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.438300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ea8d5-d3fa-460e-ac52-38bec4107ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.440247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ea8d5-d3fa-460e-ac52-38bec4107ec9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.445025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 789/1583] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.445025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9ab5e-f772-4779-b61c-6c232e264bc9 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.455034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:53.456320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9ab5e-f772-4779-b61c-6c232e264bc9 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.461865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c750249-c15b-43f9-b227-b8bba0c49d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.462543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c750249-c15b-43f9-b227-b8bba0c49d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.462817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c750249-c15b-43f9-b227-b8bba0c49d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.463082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c750249-c15b-43f9-b227-b8bba0c49d08 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.463700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c750249-c15b-43f9-b227-b8bba0c49d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.465623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c750249-c15b-43f9-b227-b8bba0c49d08 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.468784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 795/1584] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.468784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ea8d5-d3fa-460e-ac52-38bec4107ec9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.476649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ea8d5-d3fa-460e-ac52-38bec4107ec9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.482457 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60081305-c8e2-41d3-8dde-dff2a873e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.483049 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60081305-c8e2-41d3-8dde-dff2a873e16f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.483329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60081305-c8e2-41d3-8dde-dff2a873e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.483597 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60081305-c8e2-41d3-8dde-dff2a873e16f admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.484239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60081305-c8e2-41d3-8dde-dff2a873e16f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.486135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60081305-c8e2-41d3-8dde-dff2a873e16f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.488937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 790/1585] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.489730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c750249-c15b-43f9-b227-b8bba0c49d08 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.500585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c750249-c15b-43f9-b227-b8bba0c49d08 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.505522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990f55d-e41d-43ba-a746-0cf6f761ebae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.506138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990f55d-e41d-43ba-a746-0cf6f761ebae admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.506416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990f55d-e41d-43ba-a746-0cf6f761ebae admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.506683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990f55d-e41d-43ba-a746-0cf6f761ebae admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.507299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990f55d-e41d-43ba-a746-0cf6f761ebae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.509418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e990f55d-e41d-43ba-a746-0cf6f761ebae admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.510131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 796/1586] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.511054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60081305-c8e2-41d3-8dde-dff2a873e16f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.531593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 791/1587] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:53.532904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990f55d-e41d-43ba-a746-0cf6f761ebae admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.538263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:53.545026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e990f55d-e41d-43ba-a746-0cf6f761ebae admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.551195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b5258-acd5-4ff1-99fb-314e1045360a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.551786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5258-acd5-4ff1-99fb-314e1045360a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.552064 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5258-acd5-4ff1-99fb-314e1045360a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.552494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b5258-acd5-4ff1-99fb-314e1045360a admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.553157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5258-acd5-4ff1-99fb-314e1045360a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.554798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b5258-acd5-4ff1-99fb-314e1045360a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.566030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 792/1588] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.567240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63fb8a-46e8-49b2-a347-40ee4ffd816c None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.567556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63fb8a-46e8-49b2-a347-40ee4ffd816c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.567798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63fb8a-46e8-49b2-a347-40ee4ffd816c None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.569041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 793/1589] 10.222.0.43 () {54 vars in 962 bytes} [Thu Aug 11 16:27:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:27:53.569740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63fb8a-46e8-49b2-a347-40ee4ffd816c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.570909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60081305-c8e2-41d3-8dde-dff2a873e16f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.574855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf4a2c-ac53-4932-9740-05227e21f47b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.575469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf4a2c-ac53-4932-9740-05227e21f47b service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.575906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf4a2c-ac53-4932-9740-05227e21f47b service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.576033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf4a2c-ac53-4932-9740-05227e21f47b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.582228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63fb8a-46e8-49b2-a347-40ee4ffd816c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.585786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf4a2c-ac53-4932-9740-05227e21f47b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.586850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241377d-009a-4462-a2cd-754df9f8a814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.587421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241377d-009a-4462-a2cd-754df9f8a814 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.587531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf4a2c-ac53-4932-9740-05227e21f47b service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.587695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241377d-009a-4462-a2cd-754df9f8a814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.587955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241377d-009a-4462-a2cd-754df9f8a814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.588671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241377d-009a-4462-a2cd-754df9f8a814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.590277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241377d-009a-4462-a2cd-754df9f8a814 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.597198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 797/1590] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:53.598286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf4a2c-ac53-4932-9740-05227e21f47b service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.604846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 794/1591] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.604846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241377d-009a-4462-a2cd-754df9f8a814 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.608859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf4a2c-ac53-4932-9740-05227e21f47b service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.613836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108634a6-e53f-454c-8348-2b08d8e70dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.614267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108634a6-e53f-454c-8348-2b08d8e70dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.614391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241377d-009a-4462-a2cd-754df9f8a814 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.614588 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108634a6-e53f-454c-8348-2b08d8e70dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.614816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108634a6-e53f-454c-8348-2b08d8e70dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.615452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108634a6-e53f-454c-8348-2b08d8e70dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.617083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108634a6-e53f-454c-8348-2b08d8e70dfb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.620687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b106c-3582-4069-9314-47262d9d928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.621306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b106c-3582-4069-9314-47262d9d928e admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.621585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b106c-3582-4069-9314-47262d9d928e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.621854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b106c-3582-4069-9314-47262d9d928e admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.622495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b106c-3582-4069-9314-47262d9d928e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.624138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b106c-3582-4069-9314-47262d9d928e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.628056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 798/1592] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.631878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108634a6-e53f-454c-8348-2b08d8e70dfb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.639038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 795/1593] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.639038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b106c-3582-4069-9314-47262d9d928e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.642241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108634a6-e53f-454c-8348-2b08d8e70dfb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.647545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309ff6c-828b-4529-8232-e98a84c32ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.648196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309ff6c-828b-4529-8232-e98a84c32ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.648483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309ff6c-828b-4529-8232-e98a84c32ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.648757 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309ff6c-828b-4529-8232-e98a84c32ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.648879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b106c-3582-4069-9314-47262d9d928e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.649553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309ff6c-828b-4529-8232-e98a84c32ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.651350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309ff6c-828b-4529-8232-e98a84c32ce2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.653564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b37af-33e6-4c5c-aaa8-0b01d47aef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.654163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b37af-33e6-4c5c-aaa8-0b01d47aef5a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.654444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b37af-33e6-4c5c-aaa8-0b01d47aef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.654713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b37af-33e6-4c5c-aaa8-0b01d47aef5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.655355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b37af-33e6-4c5c-aaa8-0b01d47aef5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.656959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b37af-33e6-4c5c-aaa8-0b01d47aef5a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.662423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 799/1594] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.663451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309ff6c-828b-4529-8232-e98a84c32ce2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.671124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 796/1595] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.672390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0e74-520c-4bfc-a62a-dc7c45588c95 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.672679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0e74-520c-4bfc-a62a-dc7c45588c95 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.672953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0e74-520c-4bfc-a62a-dc7c45588c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.677103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309ff6c-828b-4529-8232-e98a84c32ce2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.684353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803ad19-e494-4d31-a2bd-aba2acaa847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.684975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803ad19-e494-4d31-a2bd-aba2acaa847c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.685256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803ad19-e494-4d31-a2bd-aba2acaa847c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.685540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803ad19-e494-4d31-a2bd-aba2acaa847c admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.686185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803ad19-e494-4d31-a2bd-aba2acaa847c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.688545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803ad19-e494-4d31-a2bd-aba2acaa847c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.705985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 800/1596] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.705985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803ad19-e494-4d31-a2bd-aba2acaa847c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.718140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803ad19-e494-4d31-a2bd-aba2acaa847c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.723836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-a7ac0e74-520c-4bfc-a62a-dc7c45588c95 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:53.724203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bc048-23da-4793-8953-6360eefe2c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.724930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bc048-23da-4793-8953-6360eefe2c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.725120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a7ac0e74-520c-4bfc-a62a-dc7c45588c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:53.725276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bc048-23da-4793-8953-6360eefe2c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.725657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bc048-23da-4793-8953-6360eefe2c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.726310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bc048-23da-4793-8953-6360eefe2c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.728760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bc048-23da-4793-8953-6360eefe2c9f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.739717 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 801/1597] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.740672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bc048-23da-4793-8953-6360eefe2c9f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.751434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bc048-23da-4793-8953-6360eefe2c9f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.756664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806a3e-2351-4ece-84b2-c443ed3a8b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.757339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806a3e-2351-4ece-84b2-c443ed3a8b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.757602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806a3e-2351-4ece-84b2-c443ed3a8b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.757838 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806a3e-2351-4ece-84b2-c443ed3a8b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.762766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0806a3e-2351-4ece-84b2-c443ed3a8b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.764428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0806a3e-2351-4ece-84b2-c443ed3a8b83 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.778568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 802/1598] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.778568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806a3e-2351-4ece-84b2-c443ed3a8b83 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.788654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806a3e-2351-4ece-84b2-c443ed3a8b83 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.794922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351917f-ff18-4755-be5f-8ab4fdbbf789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.796281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351917f-ff18-4755-be5f-8ab4fdbbf789 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.796504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351917f-ff18-4755-be5f-8ab4fdbbf789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.796745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351917f-ff18-4755-be5f-8ab4fdbbf789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.797557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351917f-ff18-4755-be5f-8ab4fdbbf789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.800378 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351917f-ff18-4755-be5f-8ab4fdbbf789 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.814143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 803/1599] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.815502 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351917f-ff18-4755-be5f-8ab4fdbbf789 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.830613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351917f-ff18-4755-be5f-8ab4fdbbf789 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.833953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 797/1600] 10.222.0.43 () {58 vars in 1059 bytes} [Thu Aug 11 16:27:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:53.834856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac0e74-520c-4bfc-a62a-dc7c45588c95 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.839414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80848e8-4b41-4ce1-9727-b9ecabafe919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.840025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80848e8-4b41-4ce1-9727-b9ecabafe919 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.840339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80848e8-4b41-4ce1-9727-b9ecabafe919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.840615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80848e8-4b41-4ce1-9727-b9ecabafe919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.841278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80848e8-4b41-4ce1-9727-b9ecabafe919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.842937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80848e8-4b41-4ce1-9727-b9ecabafe919 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.846927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac0e74-520c-4bfc-a62a-dc7c45588c95 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.851658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e123f30-b911-4db7-91e1-08efcfb23af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.852555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e123f30-b911-4db7-91e1-08efcfb23af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.852918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e123f30-b911-4db7-91e1-08efcfb23af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.853266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e123f30-b911-4db7-91e1-08efcfb23af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.853719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 804/1601] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.854423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80848e8-4b41-4ce1-9727-b9ecabafe919 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.857190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e123f30-b911-4db7-91e1-08efcfb23af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.859325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e123f30-b911-4db7-91e1-08efcfb23af7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.865395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80848e8-4b41-4ce1-9727-b9ecabafe919 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.871211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddde77c-87e7-4141-b894-e4733945e06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.871860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddde77c-87e7-4141-b894-e4733945e06f admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.871993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 798/1602] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:53.872214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddde77c-87e7-4141-b894-e4733945e06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.872443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddde77c-87e7-4141-b894-e4733945e06f admin admin] PATH_INFO: `/v3/projects/d5eaa0f11e014b5ab1a0e95710ede90a/users/f3a51c3ace4c45fcb9ecb3e9d98fa216/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.872725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e123f30-b911-4db7-91e1-08efcfb23af7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.888015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e123f30-b911-4db7-91e1-08efcfb23af7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:53.888763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddde77c-87e7-4141-b894-e4733945e06f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5eaa0f11e014b5ab1a0e95710ede90a, user_id=f3a51c3ace4c45fcb9ecb3e9d98fa216, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.891986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddde77c-87e7-4141-b894-e4733945e06f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.893155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091adab-0750-40ab-b89f-5888dc8d5227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:53.894057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091adab-0750-40ab-b89f-5888dc8d5227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:53.894345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091adab-0750-40ab-b89f-5888dc8d5227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:53.894620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091adab-0750-40ab-b89f-5888dc8d5227 admin admin] PATH_INFO: `/v3/projects/77b79bd8e6b84cd3a082d80b4aa3a546/users/eca40eca0b6140a498e034e3e37a8c47/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:53.910812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 805/1603] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:53 2022] PUT /identity/v3/projects/d5eaa0f11e014b5ab1a0e95710ede90a/users/f3a51c3ace4c45fcb9ecb3e9d98fa216/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:53.911824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddde77c-87e7-4141-b894-e4733945e06f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.922485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091adab-0750-40ab-b89f-5888dc8d5227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b79bd8e6b84cd3a082d80b4aa3a546, user_id=eca40eca0b6140a498e034e3e37a8c47, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:53.926541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091adab-0750-40ab-b89f-5888dc8d5227 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:53.945535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 799/1604] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:53 2022] PUT /identity/v3/projects/77b79bd8e6b84cd3a082d80b4aa3a546/users/eca40eca0b6140a498e034e3e37a8c47/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:53.947323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091adab-0750-40ab-b89f-5888dc8d5227 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:53.965318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:53.989768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddde77c-87e7-4141-b894-e4733945e06f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:54.016219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091adab-0750-40ab-b89f-5888dc8d5227 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:54.021166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3944a7-5ea6-462a-a16d-4f82a9397da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:54.022114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3944a7-5ea6-462a-a16d-4f82a9397da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.022395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3944a7-5ea6-462a-a16d-4f82a9397da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.022661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3944a7-5ea6-462a-a16d-4f82a9397da3 admin admin] PATH_INFO: `/v3/projects/1882257ce68f4b75b18946c697816ef7/users/3de4765f3004489b853e855337da820c/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.027347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:54.044766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3944a7-5ea6-462a-a16d-4f82a9397da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1882257ce68f4b75b18946c697816ef7, user_id=3de4765f3004489b853e855337da820c, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:54.048413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3944a7-5ea6-462a-a16d-4f82a9397da3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:54.065957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81138ebf-297e-4a9d-9ef4-7be83b89b090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:54.067078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81138ebf-297e-4a9d-9ef4-7be83b89b090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.067502 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81138ebf-297e-4a9d-9ef4-7be83b89b090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.067924 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81138ebf-297e-4a9d-9ef4-7be83b89b090 admin admin] PATH_INFO: `/v3/projects/fa83645c0fed4133b9eaf4c67e2bbad8/users/60ee85a52c0244879f10e8943d20dfb8/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.089106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81138ebf-297e-4a9d-9ef4-7be83b89b090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa83645c0fed4133b9eaf4c67e2bbad8, user_id=60ee85a52c0244879f10e8943d20dfb8, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:54.089350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 800/1605] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:53 2022] PUT /identity/v3/projects/1882257ce68f4b75b18946c697816ef7/users/3de4765f3004489b853e855337da820c/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:54.090748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3944a7-5ea6-462a-a16d-4f82a9397da3 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:54.092890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81138ebf-297e-4a9d-9ef4-7be83b89b090 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:54.111259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 806/1606] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:53 2022] PUT /identity/v3/projects/fa83645c0fed4133b9eaf4c67e2bbad8/users/60ee85a52c0244879f10e8943d20dfb8/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:54.116462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81138ebf-297e-4a9d-9ef4-7be83b89b090 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:54.178230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3944a7-5ea6-462a-a16d-4f82a9397da3 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:54.203254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81138ebf-297e-4a9d-9ef4-7be83b89b090 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:54.209808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62122e81-32d2-4c17-bba2-4c86e097f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:54.210602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122e81-32d2-4c17-bba2-4c86e097f8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.210884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122e81-32d2-4c17-bba2-4c86e097f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.211472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122e81-32d2-4c17-bba2-4c86e097f8a5 admin admin] PATH_INFO: `/v3/projects/0ea665a47e44438aa8c4a9e763c78fdd/users/5d59680b3fde47bf81c7765ca4d8a757/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.233560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62122e81-32d2-4c17-bba2-4c86e097f8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea665a47e44438aa8c4a9e763c78fdd, user_id=5d59680b3fde47bf81c7765ca4d8a757, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:54.236681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62122e81-32d2-4c17-bba2-4c86e097f8a5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:54.258186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 807/1607] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:54 2022] PUT /identity/v3/projects/0ea665a47e44438aa8c4a9e763c78fdd/users/5d59680b3fde47bf81c7765ca4d8a757/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:54.262382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62122e81-32d2-4c17-bba2-4c86e097f8a5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:54.266986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf982db3-e8b1-4d10-b5e7-40291d591a8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:54.267772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf982db3-e8b1-4d10-b5e7-40291d591a8b service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.268052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf982db3-e8b1-4d10-b5e7-40291d591a8b service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.268375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf982db3-e8b1-4d10-b5e7-40291d591a8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.326858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:54.339692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62122e81-32d2-4c17-bba2-4c86e097f8a5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:54.344437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf982db3-e8b1-4d10-b5e7-40291d591a8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:54.345100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893af04-d8a4-410a-9dd2-3cbb6afe2c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:54.345780 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893af04-d8a4-410a-9dd2-3cbb6afe2c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.346088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893af04-d8a4-410a-9dd2-3cbb6afe2c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.346205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf982db3-e8b1-4d10-b5e7-40291d591a8b service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:54.346387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893af04-d8a4-410a-9dd2-3cbb6afe2c83 admin admin] PATH_INFO: `/v3/projects/91ed4e8fe7284bd5863ba22aa552e60b/users/e7e5b04101c1403d9b3a311803ac798f/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.357310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 801/1608] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:54.374007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893af04-d8a4-410a-9dd2-3cbb6afe2c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ed4e8fe7284bd5863ba22aa552e60b, user_id=e7e5b04101c1403d9b3a311803ac798f, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:54.376958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893af04-d8a4-410a-9dd2-3cbb6afe2c83 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:54.377566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2799-51b7-4e74-a9f3-e454bbbc97fe None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.377852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2799-51b7-4e74-a9f3-e454bbbc97fe None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.378122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb2799-51b7-4e74-a9f3-e454bbbc97fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.394965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 808/1609] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:54 2022] PUT /identity/v3/projects/91ed4e8fe7284bd5863ba22aa552e60b/users/e7e5b04101c1403d9b3a311803ac798f/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:54.396369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dda08-0810-4999-9705-6f9dc0d8633b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.396690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dda08-0810-4999-9705-6f9dc0d8633b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.397047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dda08-0810-4999-9705-6f9dc0d8633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.412359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-51cb2799-51b7-4e74-a9f3-e454bbbc97fe None None] MFA Rules not processed for user `f3a51c3ace4c45fcb9ecb3e9d98fa216`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:54.414246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-51cb2799-51b7-4e74-a9f3-e454bbbc97fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:54.418834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-952dda08-0810-4999-9705-6f9dc0d8633b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:54.420706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-952dda08-0810-4999-9705-6f9dc0d8633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:54.436480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:54.513023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 809/1610] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:54.513023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06884030-4c83-4986-880f-bb5957ac9c16 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.513023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06884030-4c83-4986-880f-bb5957ac9c16 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.513023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06884030-4c83-4986-880f-bb5957ac9c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.551490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-06884030-4c83-4986-880f-bb5957ac9c16 None None] MFA Rules not processed for user `3de4765f3004489b853e855337da820c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:54.553136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-06884030-4c83-4986-880f-bb5957ac9c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:54.585090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 802/1611] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:54.587788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b358f5e-3ecd-4316-ac28-ab8a7bc5a990 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.588369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b358f5e-3ecd-4316-ac28-ab8a7bc5a990 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.589027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b358f5e-3ecd-4316-ac28-ab8a7bc5a990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.611091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4b358f5e-3ecd-4316-ac28-ab8a7bc5a990 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:54.612881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4b358f5e-3ecd-4316-ac28-ab8a7bc5a990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:54.623796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 803/1612] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:54.625474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785824-07a3-43bd-9a0a-070d070914fe None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.626087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785824-07a3-43bd-9a0a-070d070914fe None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.626578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785824-07a3-43bd-9a0a-070d070914fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.652530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f6785824-07a3-43bd-9a0a-070d070914fe None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:54.654002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f6785824-07a3-43bd-9a0a-070d070914fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:54.662919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 804/1613] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:54.664491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb21dc-f69c-44fa-bec6-b468143faa93 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.665575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb21dc-f69c-44fa-bec6-b468143faa93 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.666169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb21dc-f69c-44fa-bec6-b468143faa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.692375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e0eb21dc-f69c-44fa-bec6-b468143faa93 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:54.693082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 810/1614] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:54.693933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e0eb21dc-f69c-44fa-bec6-b468143faa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:54.694070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06884030-4c83-4986-880f-bb5957ac9c16 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:54.695472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-06884030-4c83-4986-880f-bb5957ac9c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:54.712347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 805/1615] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:54.713400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb21dc-f69c-44fa-bec6-b468143faa93 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:54.735587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb21dc-f69c-44fa-bec6-b468143faa93 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:54.748625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:54.749502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.749851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.750190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.751796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:54.754778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:54.791700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 806/1616] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:54 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:54.793158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:54.796723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:54.806055 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06884030-4c83-4986-880f-bb5957ac9c16 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:54.831731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:54.874484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee5818-b0a0-4e5f-aafd-08a2098b77d0 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:54.878490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8455d0-1f6f-4f19-b3fc-84ce85f9f1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:54.879103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8455d0-1f6f-4f19-b3fc-84ce85f9f1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.879387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8455d0-1f6f-4f19-b3fc-84ce85f9f1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.879659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8455d0-1f6f-4f19-b3fc-84ce85f9f1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.880859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb131b4-9772-4d71-899a-b3fa01a5173e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:54.881477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb131b4-9772-4d71-899a-b3fa01a5173e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:54.881753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb131b4-9772-4d71-899a-b3fa01a5173e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:54.882020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb131b4-9772-4d71-899a-b3fa01a5173e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:54.972124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8455d0-1f6f-4f19-b3fc-84ce85f9f1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:54.974589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8455d0-1f6f-4f19-b3fc-84ce85f9f1f0 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:54.976908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb131b4-9772-4d71-899a-b3fa01a5173e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:54.979232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb131b4-9772-4d71-899a-b3fa01a5173e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:54.989181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 807/1617] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:54.989988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8455d0-1f6f-4f19-b3fc-84ce85f9f1f0 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.000909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8455d0-1f6f-4f19-b3fc-84ce85f9f1f0 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.005816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 811/1618] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:55.006467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c97949-a3b6-4559-95ab-e5aadde1c478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.007266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c97949-a3b6-4559-95ab-e5aadde1c478 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.007414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c97949-a3b6-4559-95ab-e5aadde1c478 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.007920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c97949-a3b6-4559-95ab-e5aadde1c478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.008087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82078159-89f1-4dd4-bc5b-6f3e361c308b None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.013866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82078159-89f1-4dd4-bc5b-6f3e361c308b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.014256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82078159-89f1-4dd4-bc5b-6f3e361c308b None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.015694 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 812/1619] 10.222.0.43 () {54 vars in 912 bytes} [Thu Aug 11 16:27:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:27:55.020670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c97949-a3b6-4559-95ab-e5aadde1c478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.023240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c97949-a3b6-4559-95ab-e5aadde1c478 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.023390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d79e60-fa9d-492f-a3f2-4342ee502537 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.023390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d79e60-fa9d-492f-a3f2-4342ee502537 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.023390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d79e60-fa9d-492f-a3f2-4342ee502537 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.026333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 813/1620] 10.222.0.43 () {54 vars in 962 bytes} [Thu Aug 11 16:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:27:55.029554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d79e60-fa9d-492f-a3f2-4342ee502537 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.035060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 808/1621] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:55.036042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c97949-a3b6-4559-95ab-e5aadde1c478 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.045207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:55.049869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c97949-a3b6-4559-95ab-e5aadde1c478 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.060061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d79e60-fa9d-492f-a3f2-4342ee502537 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.064370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf5dfd-35d5-4aff-9bf8-9fe6298aa0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.066452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf5dfd-35d5-4aff-9bf8-9fe6298aa0bf admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.067346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf5dfd-35d5-4aff-9bf8-9fe6298aa0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.067880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff549175-3f26-47e4-8f22-390f8aa68d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.068792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549175-3f26-47e4-8f22-390f8aa68d5b service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.068989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf5dfd-35d5-4aff-9bf8-9fe6298aa0bf admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.069471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549175-3f26-47e4-8f22-390f8aa68d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.069898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff549175-3f26-47e4-8f22-390f8aa68d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.070657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf5dfd-35d5-4aff-9bf8-9fe6298aa0bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.075918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf5dfd-35d5-4aff-9bf8-9fe6298aa0bf admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.083281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff549175-3f26-47e4-8f22-390f8aa68d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.085514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff549175-3f26-47e4-8f22-390f8aa68d5b service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.098595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 814/1622] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:55.101826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b007ef-2346-41c9-a79e-800f9921ff27 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.103155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b007ef-2346-41c9-a79e-800f9921ff27 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.103703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b007ef-2346-41c9-a79e-800f9921ff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.139631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c8b007ef-2346-41c9-a79e-800f9921ff27 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:55.143207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 809/1623] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:27:55 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:27:55.143318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c8b007ef-2346-41c9-a79e-800f9921ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:55.143446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c83e2-c904-4a7b-92fe-8ff3b474d476 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.143446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c83e2-c904-4a7b-92fe-8ff3b474d476 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.143446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c83e2-c904-4a7b-92fe-8ff3b474d476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.178855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-041c83e2-c904-4a7b-92fe-8ff3b474d476 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:55.182715 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-041c83e2-c904-4a7b-92fe-8ff3b474d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:55.273698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 815/1624] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:55.275087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b007ef-2346-41c9-a79e-800f9921ff27 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.296452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b007ef-2346-41c9-a79e-800f9921ff27 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.302272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1c24-e584-4256-9218-7d8b24ade2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.303064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1c24-e584-4256-9218-7d8b24ade2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.303485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1c24-e584-4256-9218-7d8b24ade2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.303760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1c24-e584-4256-9218-7d8b24ade2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.304766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e1c24-e584-4256-9218-7d8b24ade2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.307509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e1c24-e584-4256-9218-7d8b24ade2e9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.321766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 816/1625] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:55.323027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1c24-e584-4256-9218-7d8b24ade2e9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.326709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 810/1626] 10.222.0.43 () {58 vars in 1059 bytes} [Thu Aug 11 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:55.328886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c83e2-c904-4a7b-92fe-8ff3b474d476 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.357373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c83e2-c904-4a7b-92fe-8ff3b474d476 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.376332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d300791-6014-4e62-a1a4-641a3df009a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.377145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d300791-6014-4e62-a1a4-641a3df009a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.378717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d300791-6014-4e62-a1a4-641a3df009a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.381260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d300791-6014-4e62-a1a4-641a3df009a2 admin admin] PATH_INFO: `/v3/projects/4a376d397619428299821c06de3d3b0e/users/29f1897c054249ae837db89c4ed7c105/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.411415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:55.426918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d300791-6014-4e62-a1a4-641a3df009a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a376d397619428299821c06de3d3b0e, user_id=29f1897c054249ae837db89c4ed7c105, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.432496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d300791-6014-4e62-a1a4-641a3df009a2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.443870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1c24-e584-4256-9218-7d8b24ade2e9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.452450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc5c47-c437-4c44-9eba-12562bf8ef93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.460134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc5c47-c437-4c44-9eba-12562bf8ef93 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.462484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc5c47-c437-4c44-9eba-12562bf8ef93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.462894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc5c47-c437-4c44-9eba-12562bf8ef93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.473309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 811/1627] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:55 2022] PUT /identity/v3/projects/4a376d397619428299821c06de3d3b0e/users/29f1897c054249ae837db89c4ed7c105/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:55.475446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d300791-6014-4e62-a1a4-641a3df009a2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.487084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc5c47-c437-4c44-9eba-12562bf8ef93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.489173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc5c47-c437-4c44-9eba-12562bf8ef93 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.506830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 817/1628] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:55.520847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc5c47-c437-4c44-9eba-12562bf8ef93 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.549178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc5c47-c437-4c44-9eba-12562bf8ef93 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.590217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d300791-6014-4e62-a1a4-641a3df009a2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.603324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5716998-4b16-4353-be04-f73258a68f70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.604644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716998-4b16-4353-be04-f73258a68f70 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.605060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716998-4b16-4353-be04-f73258a68f70 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.605430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5716998-4b16-4353-be04-f73258a68f70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.662919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffe4db-6a27-46ed-805e-06b5e7360df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.663593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffe4db-6a27-46ed-805e-06b5e7360df4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.663876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffe4db-6a27-46ed-805e-06b5e7360df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.664171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffe4db-6a27-46ed-805e-06b5e7360df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.665796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffe4db-6a27-46ed-805e-06b5e7360df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.667436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffe4db-6a27-46ed-805e-06b5e7360df4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.673357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:55.686336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5716998-4b16-4353-be04-f73258a68f70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.688081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5716998-4b16-4353-be04-f73258a68f70 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.691622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 818/1629] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:27:55.693281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffe4db-6a27-46ed-805e-06b5e7360df4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:55.715936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 812/1630] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:55.718143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffe4db-6a27-46ed-805e-06b5e7360df4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:55.724058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19ba37-bb46-4565-bb5e-d317f3c50cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:55.725786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19ba37-bb46-4565-bb5e-d317f3c50cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.726229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19ba37-bb46-4565-bb5e-d317f3c50cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.726659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a19ba37-bb46-4565-bb5e-d317f3c50cc8 admin admin] PATH_INFO: `/v3/projects/4a376d397619428299821c06de3d3b0e/users/29f1897c054249ae837db89c4ed7c105/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.729982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19ba37-bb46-4565-bb5e-d317f3c50cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a376d397619428299821c06de3d3b0e, user_id=29f1897c054249ae837db89c4ed7c105, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:55.733017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a19ba37-bb46-4565-bb5e-d317f3c50cc8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:55.766544 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 819/1631] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:27:55 2022] PUT /identity/v3/projects/4a376d397619428299821c06de3d3b0e/users/29f1897c054249ae837db89c4ed7c105/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:55.777282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f2741-0c98-4b45-b7eb-81686c1dd8eb None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:55.777577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f2741-0c98-4b45-b7eb-81686c1dd8eb None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:55.777858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f2741-0c98-4b45-b7eb-81686c1dd8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:55.857792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-720f2741-0c98-4b45-b7eb-81686c1dd8eb None None] MFA Rules not processed for user `29f1897c054249ae837db89c4ed7c105`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:55.858287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-720f2741-0c98-4b45-b7eb-81686c1dd8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:56.001627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 813/1632] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:27:56.017208 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19ba37-bb46-4565-bb5e-d317f3c50cc8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:56.153589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a19ba37-bb46-4565-bb5e-d317f3c50cc8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:56.162474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595e6ac-b970-463d-855f-2f229a7a24a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:56.163113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595e6ac-b970-463d-855f-2f229a7a24a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:56.164571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595e6ac-b970-463d-855f-2f229a7a24a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:56.164991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595e6ac-b970-463d-855f-2f229a7a24a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:56.188443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595e6ac-b970-463d-855f-2f229a7a24a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:56.191784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595e6ac-b970-463d-855f-2f229a7a24a5 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:56.214731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 820/1633] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:56.334440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67123e-3dd9-4c07-b0ff-e09402e62c5a None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:56.334819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67123e-3dd9-4c07-b0ff-e09402e62c5a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:56.335161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67123e-3dd9-4c07-b0ff-e09402e62c5a None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:56.337432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 814/1634] 10.222.0.43 () {54 vars in 912 bytes} [Thu Aug 11 16:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:27:56.348484 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50440d6-d676-4024-96ce-dc49575ced0f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:56.348773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50440d6-d676-4024-96ce-dc49575ced0f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:56.349090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50440d6-d676-4024-96ce-dc49575ced0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:56.431056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f50440d6-d676-4024-96ce-dc49575ced0f None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:56.439277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f50440d6-d676-4024-96ce-dc49575ced0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:56.614058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:56.662742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 821/1635] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:27:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:27:56.676840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67123e-3dd9-4c07-b0ff-e09402e62c5a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:56.699969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67123e-3dd9-4c07-b0ff-e09402e62c5a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:56.715745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb94c2-04a2-4260-8f27-9e822130b5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:56.718723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb94c2-04a2-4260-8f27-9e822130b5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:56.723529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb94c2-04a2-4260-8f27-9e822130b5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:56.724218 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb94c2-04a2-4260-8f27-9e822130b5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:56.752168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb94c2-04a2-4260-8f27-9e822130b5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:56.754595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb94c2-04a2-4260-8f27-9e822130b5f0 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:56.811511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 815/1636] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:56.995741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50440d6-d676-4024-96ce-dc49575ced0f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:57.023593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50440d6-d676-4024-96ce-dc49575ced0f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:57.036524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f1897c054249ae837db89c4ed7c105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a376d397619428299821c06de3d3b0e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:57.038911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:57.040703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:57.041251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:57.042176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:57.042840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:57.050406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:57.069748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb94c2-04a2-4260-8f27-9e822130b5f0 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:57.103973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 822/1637] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:27:57.111619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:57.141661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61487b-80b5-44a8-b090-7fb7015681d7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:57.152296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d01477-ccc5-4cb2-898e-a768b7b721f7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f1897c054249ae837db89c4ed7c105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a376d397619428299821c06de3d3b0e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:57.153006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d01477-ccc5-4cb2-898e-a768b7b721f7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:57.153362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d01477-ccc5-4cb2-898e-a768b7b721f7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:57.153720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d01477-ccc5-4cb2-898e-a768b7b721f7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:57.154790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d01477-ccc5-4cb2-898e-a768b7b721f7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:57.157181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d01477-ccc5-4cb2-898e-a768b7b721f7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:57.198707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 823/1638] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:57 2022] POST /identity/v3/projects => generated 354 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:27:57.199416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb94c2-04a2-4260-8f27-9e822130b5f0 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:57.203411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d01477-ccc5-4cb2-898e-a768b7b721f7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:57.309146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d01477-ccc5-4cb2-898e-a768b7b721f7 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:57.316049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:57.318590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:57.318590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:57.318590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] PATH_INFO: `/v3/users/3de4765f3004489b853e855337da820c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:57.321296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de4765f3004489b853e855337da820c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:57.322735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f1897c054249ae837db89c4ed7c105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a376d397619428299821c06de3d3b0e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:57.323345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: WARNING py.warnings [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:27:57.323345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: warnings.warn(msg) Aug 11 16:27:57.323345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: Aug 11 16:27:57.325826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:57.326310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:57.327062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:57.327488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:57.332093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:57.333198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:57.336357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:57.352985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:57.373122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 824/1639] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:27:57.433535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:57.448581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8113b-fefe-4f79-a5a4-9acd747d8799 tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:57.449897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de4765f3004489b853e855337da820c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.461245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f1897c054249ae837db89c4ed7c105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a376d397619428299821c06de3d3b0e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:57.462791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:57.463064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:57.463344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] PATH_INFO: `/v3/projects/de03c8ea46d94888a5aa20e7f9ffd602` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:57.469564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Authorizing `identity:delete_project(project_id=de03c8ea46d94888a5aa20e7f9ffd602)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:57.469841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: WARNING py.warnings [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 11 16:27:57.469841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: warnings.warn(msg) Aug 11 16:27:57.469841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: Aug 11 16:27:57.474532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:57.485469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de4765f3004489b853e855337da820c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.486404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de4765f3004489b853e855337da820c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.504748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de4765f3004489b853e855337da820c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.519130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a3b956e5-ab32-4cd1-8d47-29582cb1bd84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de4765f3004489b853e855337da820c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.535735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 816/1640] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:27:57 2022] DELETE /identity/v3/users/3de4765f3004489b853e855337da820c => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:57.538030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae992f-6c1f-41a4-bf00-241bc725c00d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:57.538030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae992f-6c1f-41a4-bf00-241bc725c00d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:57.538352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae992f-6c1f-41a4-bf00-241bc725c00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:57.561837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f2ae992f-6c1f-41a4-bf00-241bc725c00d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:57.563632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f2ae992f-6c1f-41a4-bf00-241bc725c00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:57.630187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de03c8ea46d94888a5aa20e7f9ffd602', 'request_id': 'req-e2950754-ca15-46e9-8f03-701902b74bad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.653909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de03c8ea46d94888a5aa20e7f9ffd602', 'request_id': 'req-e2950754-ca15-46e9-8f03-701902b74bad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.669275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 817/1641] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:57.702920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de03c8ea46d94888a5aa20e7f9ffd602', 'request_id': 'req-e2950754-ca15-46e9-8f03-701902b74bad'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.727951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] The token cache is being invalidate because project de03c8ea46d94888a5aa20e7f9ffd602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:27:57.728410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.730521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:57.733011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 825/1642] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:27:57 2022] DELETE /identity/v3/projects/de03c8ea46d94888a5aa20e7f9ffd602 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:27:57.736160 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae992f-6c1f-41a4-bf00-241bc725c00d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:57.738134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f2ae992f-6c1f-41a4-bf00-241bc725c00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:57.902953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae992f-6c1f-41a4-bf00-241bc725c00d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:57.913478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5a254-f315-4252-81e0-2fa078807af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:57.914119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5a254-f315-4252-81e0-2fa078807af8 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:57.914411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5a254-f315-4252-81e0-2fa078807af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:57.914890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5a254-f315-4252-81e0-2fa078807af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:57.918002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-62b5a254-f315-4252-81e0-2fa078807af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:57.944138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5a254-f315-4252-81e0-2fa078807af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:57.948357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5a254-f315-4252-81e0-2fa078807af8 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:57.972383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 818/1643] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:27:58.259091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:58.260867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:58.287185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2950754-ca15-46e9-8f03-701902b74bad tempest-VolumeQuotaClassesTest-375534949 tempest-VolumeQuotaClassesTest-375534949-project-admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:58.296970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:58.297630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:58.297911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:58.298182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] PATH_INFO: `/v3/users/f3a51c3ace4c45fcb9ecb3e9d98fa216` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:58.306736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a51c3ace4c45fcb9ecb3e9d98fa216)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:58.308755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:58.416268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:58.430083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a51c3ace4c45fcb9ecb3e9d98fa216'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:58.445205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a51c3ace4c45fcb9ecb3e9d98fa216'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:58.446184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a51c3ace4c45fcb9ecb3e9d98fa216'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:58.474814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a51c3ace4c45fcb9ecb3e9d98fa216'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:58.476521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a51c3ace4c45fcb9ecb3e9d98fa216'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:58.507203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 826/1644] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:27:58 2022] DELETE /identity/v3/users/f3a51c3ace4c45fcb9ecb3e9d98fa216 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:58.508868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f13b-4dd3-4fcf-8bea-eae152846ee6 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:58.509379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f13b-4dd3-4fcf-8bea-eae152846ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:58.509488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f13b-4dd3-4fcf-8bea-eae152846ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:58.532765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e473f13b-4dd3-4fcf-8bea-eae152846ee6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:58.534137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e473f13b-4dd3-4fcf-8bea-eae152846ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:58.609413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:27:58.677671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 819/1645] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:58.730249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:58.731772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:58.866382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2beec91-f797-409a-b3fc-0e4885aa06d2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:58.872541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7a0f7-fcf2-4b6c-a436-f44976afe644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:58.873168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7a0f7-fcf2-4b6c-a436-f44976afe644 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:58.873582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7a0f7-fcf2-4b6c-a436-f44976afe644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:58.873870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7a0f7-fcf2-4b6c-a436-f44976afe644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:58.889745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7a0f7-fcf2-4b6c-a436-f44976afe644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:58.893094 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7a0f7-fcf2-4b6c-a436-f44976afe644 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:58.929322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 827/1646] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:27:59.519840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473f13b-4dd3-4fcf-8bea-eae152846ee6 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:59.523321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e473f13b-4dd3-4fcf-8bea-eae152846ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:59.551707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473f13b-4dd3-4fcf-8bea-eae152846ee6 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:59.566649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:59.569719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:59.571242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:59.572284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] PATH_INFO: `/v3/projects/1882257ce68f4b75b18946c697816ef7` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:59.573955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1882257ce68f4b75b18946c697816ef7)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:59.577238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:59.725669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1882257ce68f4b75b18946c697816ef7', 'request_id': 'req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:59.739570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1882257ce68f4b75b18946c697816ef7', 'request_id': 'req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:59.749004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1882257ce68f4b75b18946c697816ef7', 'request_id': 'req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:59.768223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] The token cache is being invalidate because project 1882257ce68f4b75b18946c697816ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:27:59.768707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:59.779897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0aa9d6e3-d1f6-482e-969d-c7dea6c1967b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:27:59.785722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 820/1647] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:27:59 2022] DELETE /identity/v3/projects/1882257ce68f4b75b18946c697816ef7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:27:59.787474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b77c9-3343-4150-80d5-85c4343d2303 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:59.787916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b77c9-3343-4150-80d5-85c4343d2303 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:59.788152 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b77c9-3343-4150-80d5-85c4343d2303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:59.821692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4e1b77c9-3343-4150-80d5-85c4343d2303 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:59.826495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4e1b77c9-3343-4150-80d5-85c4343d2303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:59.908046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 828/1648] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:59.916952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafcf43-66b9-499b-b408-0e5790256003 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:59.917243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafcf43-66b9-499b-b408-0e5790256003 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:59.917524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafcf43-66b9-499b-b408-0e5790256003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:59.942821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ebafcf43-66b9-499b-b408-0e5790256003 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:27:59.944268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ebafcf43-66b9-499b-b408-0e5790256003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:27:59.953594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 821/1649] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:27:59.957586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b77c9-3343-4150-80d5-85c4343d2303 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:27:59.969183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b77c9-3343-4150-80d5-85c4343d2303 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:27:59.974725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:27:59.975302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:27:59.975835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:27:59.976299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:27:59.977892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:27:59.978762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:27:59.980295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:27:59.996793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 829/1650] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:28:00.001332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafcf43-66b9-499b-b408-0e5790256003 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:00.012623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafcf43-66b9-499b-b408-0e5790256003 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:00.019123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d4c9a-91dd-464d-9ac1-9033b0833f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:00.019843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4c9a-91dd-464d-9ac1-9033b0833f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:00.020301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4c9a-91dd-464d-9ac1-9033b0833f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:00.020650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4c9a-91dd-464d-9ac1-9033b0833f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:00.021370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d4c9a-91dd-464d-9ac1-9033b0833f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:00.024325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d4c9a-91dd-464d-9ac1-9033b0833f0c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:00.069230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 822/1651] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:27:59 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:00.074793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:00.100011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b5ec-b234-4c01-ac00-3a410ccaa20e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:00.234042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:00.283580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3e8fb-cfa4-41e9-bcb9-b8f011da9e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:00.284446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e8fb-cfa4-41e9-bcb9-b8f011da9e54 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:00.284745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e8fb-cfa4-41e9-bcb9-b8f011da9e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:00.285024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e8fb-cfa4-41e9-bcb9-b8f011da9e54 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:00.285667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3e8fb-cfa4-41e9-bcb9-b8f011da9e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:00.288175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea3e8fb-cfa4-41e9-bcb9-b8f011da9e54 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:00.368919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 830/1652] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:00 2022] POST /identity/v3/users => generated 290 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:00.371147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d4c9a-91dd-464d-9ac1-9033b0833f0c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:00.394626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d4c9a-91dd-464d-9ac1-9033b0833f0c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:00.416981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c5f0c-b2df-4dfc-8863-03211cd44920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:00.417683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c5f0c-b2df-4dfc-8863-03211cd44920 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:00.417973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c5f0c-b2df-4dfc-8863-03211cd44920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:00.418247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c5f0c-b2df-4dfc-8863-03211cd44920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:00.418965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c5f0c-b2df-4dfc-8863-03211cd44920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:00.420626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c5f0c-b2df-4dfc-8863-03211cd44920 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:00.459246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 823/1653] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:00.463992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3e8fb-cfa4-41e9-bcb9-b8f011da9e54 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:00.486862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3e8fb-cfa4-41e9-bcb9-b8f011da9e54 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:00.506181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425204d-63c0-4b8b-863a-1b3386667e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:00.506841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425204d-63c0-4b8b-863a-1b3386667e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:00.507124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425204d-63c0-4b8b-863a-1b3386667e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:00.507399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425204d-63c0-4b8b-863a-1b3386667e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:00.508652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425204d-63c0-4b8b-863a-1b3386667e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:00.511677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425204d-63c0-4b8b-863a-1b3386667e3d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:00.531492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 831/1654] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:00.536136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c5f0c-b2df-4dfc-8863-03211cd44920 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:00.551376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c5f0c-b2df-4dfc-8863-03211cd44920 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:00.564326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3ad3-7e46-4f0b-b363-79ff62ad1c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:00.565489 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3ad3-7e46-4f0b-b363-79ff62ad1c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:00.565723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3ad3-7e46-4f0b-b363-79ff62ad1c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:00.566000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f3ad3-7e46-4f0b-b363-79ff62ad1c9e admin admin] PATH_INFO: `/v3/projects/b70ab251b5d64351a2fe0cf171ef6f26/users/6aa18ff44d5e4efda91a7fe889a456bc/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:00.626670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f3ad3-7e46-4f0b-b363-79ff62ad1c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70ab251b5d64351a2fe0cf171ef6f26, user_id=6aa18ff44d5e4efda91a7fe889a456bc, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:00.629822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f3ad3-7e46-4f0b-b363-79ff62ad1c9e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:00.653080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 824/1655] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:00 2022] PUT /identity/v3/projects/b70ab251b5d64351a2fe0cf171ef6f26/users/6aa18ff44d5e4efda91a7fe889a456bc/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:00.669145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa567-7fbf-4e44-9819-4abe6029f00e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:00.669145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa567-7fbf-4e44-9819-4abe6029f00e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:00.669145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4aa567-7fbf-4e44-9819-4abe6029f00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:00.720504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7b4aa567-7fbf-4e44-9819-4abe6029f00e None None] MFA Rules not processed for user `6aa18ff44d5e4efda91a7fe889a456bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:00.722049 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7b4aa567-7fbf-4e44-9819-4abe6029f00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:00.901513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 832/1656] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:00.907846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3ad3-7e46-4f0b-b363-79ff62ad1c9e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:01.091787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:01.110128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f3ad3-7e46-4f0b-b363-79ff62ad1c9e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:01.121825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58a443-7e65-416b-ada8-3949435f1390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:01.123401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58a443-7e65-416b-ada8-3949435f1390 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:01.124590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58a443-7e65-416b-ada8-3949435f1390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:01.124875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58a443-7e65-416b-ada8-3949435f1390 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:01.125545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58a443-7e65-416b-ada8-3949435f1390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:01.129018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58a443-7e65-416b-ada8-3949435f1390 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:01.192655 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 825/1657] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:00 2022] POST /identity/v3/projects => generated 381 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:01.204447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aa567-7fbf-4e44-9819-4abe6029f00e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:01.229612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4aa567-7fbf-4e44-9819-4abe6029f00e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:01.355946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:01.356633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:01.356910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:01.357186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:01.357829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:01.359786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:01.443625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 833/1658] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:01 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:01.443848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58a443-7e65-416b-ada8-3949435f1390 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:01.460444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58a443-7e65-416b-ada8-3949435f1390 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:01.471628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deab947-4e04-4157-a271-9c7012655e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:01.472194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deab947-4e04-4157-a271-9c7012655e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:01.472387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deab947-4e04-4157-a271-9c7012655e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:01.472724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deab947-4e04-4157-a271-9c7012655e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:01.473305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deab947-4e04-4157-a271-9c7012655e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:01.479956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deab947-4e04-4157-a271-9c7012655e3f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:01.483981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:01.486900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:01.514289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 826/1659] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:01.515223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deab947-4e04-4157-a271-9c7012655e3f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:01.524862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f0f39-eaf3-47e8-ab62-b3086b8ab448 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:01.530754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deab947-4e04-4157-a271-9c7012655e3f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:01.532379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:01.533795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:01.534272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:01.534732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] PATH_INFO: `/v3/projects/d5eaa0f11e014b5ab1a0e95710ede90a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:01.536244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5eaa0f11e014b5ab1a0e95710ede90a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:01.544026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0e68-a613-4916-aeb1-6a4686feb392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:01.544616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0e68-a613-4916-aeb1-6a4686feb392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:01.544786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0e68-a613-4916-aeb1-6a4686feb392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:01.545061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc0e68-a613-4916-aeb1-6a4686feb392 admin admin] PATH_INFO: `/v3/projects/58a700e918a84de49b7002d1d2530779/users/215e9785968f41668aaf20d95ef4a9bd/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:01.546273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:01.585719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc0e68-a613-4916-aeb1-6a4686feb392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a700e918a84de49b7002d1d2530779, user_id=215e9785968f41668aaf20d95ef4a9bd, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:01.589440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc0e68-a613-4916-aeb1-6a4686feb392 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:01.628562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 827/1660] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:01 2022] PUT /identity/v3/projects/58a700e918a84de49b7002d1d2530779/users/215e9785968f41668aaf20d95ef4a9bd/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:01.630070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0e68-a613-4916-aeb1-6a4686feb392 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:01.667485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:01.692067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5eaa0f11e014b5ab1a0e95710ede90a', 'request_id': 'req-8c44b80b-1e34-4628-ae27-54cba46ef1bd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:01.724047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5eaa0f11e014b5ab1a0e95710ede90a', 'request_id': 'req-8c44b80b-1e34-4628-ae27-54cba46ef1bd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:01.760036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5eaa0f11e014b5ab1a0e95710ede90a', 'request_id': 'req-8c44b80b-1e34-4628-ae27-54cba46ef1bd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:01.803412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] The token cache is being invalidate because project d5eaa0f11e014b5ab1a0e95710ede90a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:01.804030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:01.810076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:01.815486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 834/1661] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:01 2022] DELETE /identity/v3/projects/d5eaa0f11e014b5ab1a0e95710ede90a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:01.823609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:01.827046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:01.956317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc0e68-a613-4916-aeb1-6a4686feb392 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:01.991198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b80b-1e34-4628-ae27-54cba46ef1bd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:02.004735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:02.005636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:02.006715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:02.007200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] PATH_INFO: `/v3/users/29f1897c054249ae837db89c4ed7c105` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:02.009467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f1897c054249ae837db89c4ed7c105)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:02.012748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:02.142337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:02.142970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:02.143243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:02.143520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:02.144583 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f1897c054249ae837db89c4ed7c105'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:02.152134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:02.170242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:02.176393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f1897c054249ae837db89c4ed7c105'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:02.178308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f1897c054249ae837db89c4ed7c105'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:02.234357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 828/1662] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 602 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:02.238821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:02.247667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:02.257770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f1897c054249ae837db89c4ed7c105'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:02.260859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a15d8db8-34a8-43b4-88f3-80306fefc536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f1897c054249ae837db89c4ed7c105'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:02.294305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d4503-b338-44fe-bcaf-953883695d65 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:02.307032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 835/1663] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:01 2022] DELETE /identity/v3/users/29f1897c054249ae837db89c4ed7c105 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:02.314121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:02.316447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:02.317179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:02.319846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] PATH_INFO: `/v3/projects/58a700e918a84de49b7002d1d2530779/users/215e9785968f41668aaf20d95ef4a9bd/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:02.323313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=58a700e918a84de49b7002d1d2530779, user_id=215e9785968f41668aaf20d95ef4a9bd, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:02.327044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:02.353457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 829/1664] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:02 2022] PUT /identity/v3/projects/58a700e918a84de49b7002d1d2530779/users/215e9785968f41668aaf20d95ef4a9bd/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:02.367079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a5e41-de1d-4707-8673-ee83b46ae53d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:02.368717 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a5e41-de1d-4707-8673-ee83b46ae53d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:02.369437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a5e41-de1d-4707-8673-ee83b46ae53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:02.440041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d74a5e41-de1d-4707-8673-ee83b46ae53d None None] MFA Rules not processed for user `215e9785968f41668aaf20d95ef4a9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:02.442685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d74a5e41-de1d-4707-8673-ee83b46ae53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:02.532693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:02.625714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 836/1665] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:02.644092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:02.650222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:02.775593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb390971-357e-4aac-8dda-6c8adc12984e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:02.780587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb390971-357e-4aac-8dda-6c8adc12984e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:02.781070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb390971-357e-4aac-8dda-6c8adc12984e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:02.810355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:02.879259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cb390971-357e-4aac-8dda-6c8adc12984e None None] MFA Rules not processed for user `3a5ab962b58f473f874bd94bfe2db6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:02.884931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cb390971-357e-4aac-8dda-6c8adc12984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:02.938242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08dcb3b-fa1b-4ad2-bfd6-ea25aec3d8dc admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:02.953354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:02.955606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:02.955948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:02.956256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:02.985319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:02.990278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:03.075389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 830/1666] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:03.113051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 837/1667] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:03.132411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:03.132411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:03.356651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb390971-357e-4aac-8dda-6c8adc12984e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:03.361642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cb390971-357e-4aac-8dda-6c8adc12984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:03.381062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb390971-357e-4aac-8dda-6c8adc12984e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:03.393585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a5e8c-e952-4a82-83a0-355504ec8416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:03.395266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a5e8c-e952-4a82-83a0-355504ec8416 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:03.395572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a5e8c-e952-4a82-83a0-355504ec8416 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:03.395852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a5e8c-e952-4a82-83a0-355504ec8416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:03.397867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e27a5e8c-e952-4a82-83a0-355504ec8416 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:03.421336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923a390-db97-48b1-ada4-e469e799daaf service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:03.443093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:03.444405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:03.444754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:03.445227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:03.457226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:03.463963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:03.497115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 831/1668] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:03.561001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a5e8c-e952-4a82-83a0-355504ec8416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:03.563426 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a5e8c-e952-4a82-83a0-355504ec8416 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:03.596605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 838/1669] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:03.819975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:03.823352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:04.006924 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69a0fa-d0cd-4ade-8465-2a0a416c06ac service nova] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:04.018974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:04.019926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:04.020393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:04.022051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] PATH_INFO: `/v3/users/aa6eadb8625649e29ef26b437fe81a39` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:04.024906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6eadb8625649e29ef26b437fe81a39)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:04.027038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:04.163245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6eadb8625649e29ef26b437fe81a39'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.213356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:04.215687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6eadb8625649e29ef26b437fe81a39'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.217768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6eadb8625649e29ef26b437fe81a39'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.241350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6eadb8625649e29ef26b437fe81a39'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.280751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6eadb8625649e29ef26b437fe81a39'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.316880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 832/1670] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:03 2022] DELETE /identity/v3/users/aa6eadb8625649e29ef26b437fe81a39 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:04.329395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3def80-8d6d-4c7b-ae4c-e2823d0c53de None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:04.330491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3def80-8d6d-4c7b-ae4c-e2823d0c53de None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:04.331407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3def80-8d6d-4c7b-ae4c-e2823d0c53de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:04.390396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4c3def80-8d6d-4c7b-ae4c-e2823d0c53de None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:04.392428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4c3def80-8d6d-4c7b-ae4c-e2823d0c53de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:04.518192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:04.535549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 839/1671] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:04.540091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:04.542253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:04.564412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8936c-e5ad-422c-ab59-77312fe3ef5c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:04.579880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:04.582179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:04.582561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:04.582899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] PATH_INFO: `/v3/projects/4a376d397619428299821c06de3d3b0e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:04.584305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a376d397619428299821c06de3d3b0e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:04.591010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:04.661804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3def80-8d6d-4c7b-ae4c-e2823d0c53de None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:04.663713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4c3def80-8d6d-4c7b-ae4c-e2823d0c53de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:04.745905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a376d397619428299821c06de3d3b0e', 'request_id': 'req-f683d519-dcbe-4974-a2cd-a4dfda361c73'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.769741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a376d397619428299821c06de3d3b0e', 'request_id': 'req-f683d519-dcbe-4974-a2cd-a4dfda361c73'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.785260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a376d397619428299821c06de3d3b0e', 'request_id': 'req-f683d519-dcbe-4974-a2cd-a4dfda361c73'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.815945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] The token cache is being invalidate because project 4a376d397619428299821c06de3d3b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:04.816705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.818127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f683d519-dcbe-4974-a2cd-a4dfda361c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:04.825625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 833/1672] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:04 2022] DELETE /identity/v3/projects/4a376d397619428299821c06de3d3b0e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:04.829373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a035-d13e-430b-9acc-5b7d9848b92c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:04.835143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a035-d13e-430b-9acc-5b7d9848b92c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:04.836394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42a035-d13e-430b-9acc-5b7d9848b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:04.846264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3def80-8d6d-4c7b-ae4c-e2823d0c53de None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:04.882471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4f42a035-d13e-430b-9acc-5b7d9848b92c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:04.884941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4f42a035-d13e-430b-9acc-5b7d9848b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:04.962211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:04.962987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:04.963401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:04.963818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:04.974829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:05.080364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 834/1673] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:05.081738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dded26f-b568-414f-b52d-260a8414a32c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:05.082306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dded26f-b568-414f-b52d-260a8414a32c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:05.082306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dded26f-b568-414f-b52d-260a8414a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:05.142338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-7dded26f-b568-414f-b52d-260a8414a32c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:05.157360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:05.158414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:05.165429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7dded26f-b568-414f-b52d-260a8414a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:05.183813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 840/1674] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:05.186723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 835/1675] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:05.193224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:05.207069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b70f4-c319-4a0a-8102-c309833d6dcf service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:05.213805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:05.214640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:05.215084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:05.215453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:05.220113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:05.220113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:05.220113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:05.239855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dded26f-b568-414f-b52d-260a8414a32c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:05.246494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 841/1676] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:28:05.253524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dded26f-b568-414f-b52d-260a8414a32c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:05.265565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e67bd-0a29-4e71-99a0-f2e58315791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:05.266231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e67bd-0a29-4e71-99a0-f2e58315791b admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:05.266876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e67bd-0a29-4e71-99a0-f2e58315791b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:05.266876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e67bd-0a29-4e71-99a0-f2e58315791b admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:05.267410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e67bd-0a29-4e71-99a0-f2e58315791b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:05.273605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e67bd-0a29-4e71-99a0-f2e58315791b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:05.365816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 836/1677] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:05 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:05.371781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:05.413445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a9fe4-1ae7-4df4-9487-1c14bcf5d4d6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:05.664550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:05.707393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557cbf6-6170-489d-99d1-2374c6352e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:05.708021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557cbf6-6170-489d-99d1-2374c6352e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:05.713226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557cbf6-6170-489d-99d1-2374c6352e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:05.713556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557cbf6-6170-489d-99d1-2374c6352e24 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:05.714296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557cbf6-6170-489d-99d1-2374c6352e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:05.721691 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557cbf6-6170-489d-99d1-2374c6352e24 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:05.766667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 842/1678] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:05 2022] POST /identity/v3/users => generated 296 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:05.771044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e67bd-0a29-4e71-99a0-f2e58315791b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:05.785271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e67bd-0a29-4e71-99a0-f2e58315791b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:05.815819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf40a06-eeb0-4519-891b-67b6873e2323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:05.815819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf40a06-eeb0-4519-891b-67b6873e2323 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:05.815819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf40a06-eeb0-4519-891b-67b6873e2323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:05.815819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf40a06-eeb0-4519-891b-67b6873e2323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:05.816134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf40a06-eeb0-4519-891b-67b6873e2323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:05.816471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf40a06-eeb0-4519-891b-67b6873e2323 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:05.866181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557cbf6-6170-489d-99d1-2374c6352e24 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:05.885855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557cbf6-6170-489d-99d1-2374c6352e24 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:05.886596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 837/1679] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:05.923386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720e7a8-b2d9-4cf3-baf1-b35f46dcc3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:05.924934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720e7a8-b2d9-4cf3-baf1-b35f46dcc3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:05.926485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720e7a8-b2d9-4cf3-baf1-b35f46dcc3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:05.926887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720e7a8-b2d9-4cf3-baf1-b35f46dcc3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:05.929141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720e7a8-b2d9-4cf3-baf1-b35f46dcc3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:05.934857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720e7a8-b2d9-4cf3-baf1-b35f46dcc3f5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:05.966124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf40a06-eeb0-4519-891b-67b6873e2323 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:05.966809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 843/1680] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:28:05.988702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf40a06-eeb0-4519-891b-67b6873e2323 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:06.011315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0590c-c638-4d03-8e25-bb3b2fd58495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:06.012482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0590c-c638-4d03-8e25-bb3b2fd58495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:06.012922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0590c-c638-4d03-8e25-bb3b2fd58495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:06.013349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0590c-c638-4d03-8e25-bb3b2fd58495 admin admin] PATH_INFO: `/v3/projects/a28ea2427c44437cae29bfe3dd4a2c82/users/bfc5b9cd2d0045c6a0113c5e7e3c6e78/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:06.076761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0590c-c638-4d03-8e25-bb3b2fd58495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28ea2427c44437cae29bfe3dd4a2c82, user_id=bfc5b9cd2d0045c6a0113c5e7e3c6e78, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:06.080663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0590c-c638-4d03-8e25-bb3b2fd58495 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:06.100454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:06.112876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 838/1681] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:05 2022] PUT /identity/v3/projects/a28ea2427c44437cae29bfe3dd4a2c82/users/bfc5b9cd2d0045c6a0113c5e7e3c6e78/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:06.122580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92962557-8d44-496a-b0b3-995b684f1fd7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:06.123182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92962557-8d44-496a-b0b3-995b684f1fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:06.123998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92962557-8d44-496a-b0b3-995b684f1fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:06.196136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-92962557-8d44-496a-b0b3-995b684f1fd7 None None] MFA Rules not processed for user `bfc5b9cd2d0045c6a0113c5e7e3c6e78`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:06.197399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-92962557-8d44-496a-b0b3-995b684f1fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:06.400813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 844/1682] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:06.401979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0590c-c638-4d03-8e25-bb3b2fd58495 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:06.525753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0590c-c638-4d03-8e25-bb3b2fd58495 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:06.538603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0525ea2-ac6a-472b-a299-1c1ef4d0151e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:06.539909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525ea2-ac6a-472b-a299-1c1ef4d0151e admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:06.540276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525ea2-ac6a-472b-a299-1c1ef4d0151e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:06.540634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0525ea2-ac6a-472b-a299-1c1ef4d0151e admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:06.541333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0525ea2-ac6a-472b-a299-1c1ef4d0151e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:06.547740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0525ea2-ac6a-472b-a299-1c1ef4d0151e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:06.621680 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92962557-8d44-496a-b0b3-995b684f1fd7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:06.633545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 839/1683] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:06 2022] POST /identity/v3/projects => generated 391 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:06.647167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92962557-8d44-496a-b0b3-995b684f1fd7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:06.741455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f6bfb-024a-4229-bc2e-53b4d8c97cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:06.741893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6bfb-024a-4229-bc2e-53b4d8c97cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:06.742712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6bfb-024a-4229-bc2e-53b4d8c97cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:06.742992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f6bfb-024a-4229-bc2e-53b4d8c97cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:06.743629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f6bfb-024a-4229-bc2e-53b4d8c97cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:06.745833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f6bfb-024a-4229-bc2e-53b4d8c97cd4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:06.786223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 845/1684] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:06 2022] POST /identity/v3/users => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:06.790835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0525ea2-ac6a-472b-a299-1c1ef4d0151e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:06.805568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0525ea2-ac6a-472b-a299-1c1ef4d0151e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:06.819827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7328f5-979c-4dfa-b419-cc12353a07dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:06.820815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7328f5-979c-4dfa-b419-cc12353a07dd admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:06.821101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7328f5-979c-4dfa-b419-cc12353a07dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:06.821425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7328f5-979c-4dfa-b419-cc12353a07dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:06.822091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7328f5-979c-4dfa-b419-cc12353a07dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:06.823918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7328f5-979c-4dfa-b419-cc12353a07dd admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:06.848040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 840/1685] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:06.852938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f6bfb-024a-4229-bc2e-53b4d8c97cd4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:06.865644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f6bfb-024a-4229-bc2e-53b4d8c97cd4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:06.878538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:06.880187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:06.880473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:06.880831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] PATH_INFO: `/v3/projects/25d5d82a8c6c4db79fce9140f0f2aae7/users/1ddbb2a95c30448193a5e746a68fd59e/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:06.924735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d5d82a8c6c4db79fce9140f0f2aae7, user_id=1ddbb2a95c30448193a5e746a68fd59e, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:06.927381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:06.967776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7328f5-979c-4dfa-b419-cc12353a07dd admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:06.977265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 846/1686] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:06 2022] PUT /identity/v3/projects/25d5d82a8c6c4db79fce9140f0f2aae7/users/1ddbb2a95c30448193a5e746a68fd59e/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:06.980468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7328f5-979c-4dfa-b419-cc12353a07dd admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:07.019852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:07.021610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:07.115196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:07.187413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee36aad-5dac-40d3-b1e7-bed182f1f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:07.188308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee36aad-5dac-40d3-b1e7-bed182f1f2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.188747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee36aad-5dac-40d3-b1e7-bed182f1f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.189163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee36aad-5dac-40d3-b1e7-bed182f1f2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.190007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee36aad-5dac-40d3-b1e7-bed182f1f2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:07.191758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee36aad-5dac-40d3-b1e7-bed182f1f2d4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:07.218339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 841/1687] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:07.219454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee36aad-5dac-40d3-b1e7-bed182f1f2d4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:07.248380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee36aad-5dac-40d3-b1e7-bed182f1f2d4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:07.251257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c4371-8b1d-4db0-bda0-706b5e5ab2ba admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:07.253601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a0b9b-dc51-4851-a470-7b65b698b0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:07.254275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a0b9b-dc51-4851-a470-7b65b698b0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.254553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a0b9b-dc51-4851-a470-7b65b698b0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.254826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a0b9b-dc51-4851-a470-7b65b698b0a6 admin admin] PATH_INFO: `/v3/projects/25d5d82a8c6c4db79fce9140f0f2aae7/users/1ddbb2a95c30448193a5e746a68fd59e/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.258434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a0b9b-dc51-4851-a470-7b65b698b0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d5d82a8c6c4db79fce9140f0f2aae7, user_id=1ddbb2a95c30448193a5e746a68fd59e, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:07.261172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a0b9b-dc51-4851-a470-7b65b698b0a6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:07.283695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:07.284671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.286261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.286837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] PATH_INFO: `/v3/projects/82d7c413ee4c438ab8223b42a3a5f692` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.290735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=82d7c413ee4c438ab8223b42a3a5f692)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:07.296770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:07.302350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 842/1688] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:07 2022] PUT /identity/v3/projects/25d5d82a8c6c4db79fce9140f0f2aae7/users/1ddbb2a95c30448193a5e746a68fd59e/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:07.303872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86661466-7066-461a-b4cb-5ecee215b3f7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.304263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86661466-7066-461a-b4cb-5ecee215b3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.304444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86661466-7066-461a-b4cb-5ecee215b3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.374040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-86661466-7066-461a-b4cb-5ecee215b3f7 None None] MFA Rules not processed for user `1ddbb2a95c30448193a5e746a68fd59e`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:07.375522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-86661466-7066-461a-b4cb-5ecee215b3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:07.439131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82d7c413ee4c438ab8223b42a3a5f692', 'request_id': 'req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:07.458645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82d7c413ee4c438ab8223b42a3a5f692', 'request_id': 'req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:07.476166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82d7c413ee4c438ab8223b42a3a5f692', 'request_id': 'req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:07.502391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:07.534345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] The token cache is being invalidate because project 82d7c413ee4c438ab8223b42a3a5f692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:07.535002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:07.537723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-95e4149f-93e9-4cf2-a2c3-3006ca9b6c8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:07.543678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 847/1689] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:07 2022] DELETE /identity/v3/projects/82d7c413ee4c438ab8223b42a3a5f692 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:07.548450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d380322-d733-4c4e-a779-67a59e854aff None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.549229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d380322-d733-4c4e-a779-67a59e854aff None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.551248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d380322-d733-4c4e-a779-67a59e854aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.581325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 843/1690] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:07.587372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86661466-7066-461a-b4cb-5ecee215b3f7 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:07.589061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-86661466-7066-461a-b4cb-5ecee215b3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:07.592652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0d380322-d733-4c4e-a779-67a59e854aff None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:07.593961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0d380322-d733-4c4e-a779-67a59e854aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:07.706869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86661466-7066-461a-b4cb-5ecee215b3f7 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:07.769921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 848/1691] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:07.777635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f346e7-80de-4358-8ec7-2d5552b8a0ab None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.778078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f346e7-80de-4358-8ec7-2d5552b8a0ab None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.778475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f346e7-80de-4358-8ec7-2d5552b8a0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.819712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:07.820676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.821330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.821728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.824731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:07.831674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f5f346e7-80de-4358-8ec7-2d5552b8a0ab None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:07.835388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f5f346e7-80de-4358-8ec7-2d5552b8a0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:07.855630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 849/1692] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:07.859468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f346e7-80de-4358-8ec7-2d5552b8a0ab None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:07.878072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f346e7-80de-4358-8ec7-2d5552b8a0ab None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:07.892423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:07.893218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.893602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.893977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.894960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:07.895133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:07.896897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:07.928607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 850/1693] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:28:07.933173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:07.958270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30ee5a-9dfb-4c7e-a470-9d14b6612f85 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:07.964772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:07.971072 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:07.973271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:07.974011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:07.974393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:07.974855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:07.975758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:07.977828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:08.006820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 844/1694] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:07 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:08.035659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 851/1695] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:07 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:08.040092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:08.063395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc7f92-be68-4815-a553-dacea01642c8 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:08.125735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:08.127735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:08.154791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:08.202829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:08.242867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f89f4-1bad-4f00-abc6-2b5b0e4017e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:08.243598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f89f4-1bad-4f00-abc6-2b5b0e4017e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:08.243876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f89f4-1bad-4f00-abc6-2b5b0e4017e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:08.244160 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f89f4-1bad-4f00-abc6-2b5b0e4017e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:08.244804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f89f4-1bad-4f00-abc6-2b5b0e4017e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:08.246733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f89f4-1bad-4f00-abc6-2b5b0e4017e7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:08.280395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a17bf-5ad5-4f96-be00-d00bbaac59a3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:08.285209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 845/1696] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:08 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:08.290840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f89f4-1bad-4f00-abc6-2b5b0e4017e7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:08.292909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff57e7c-a471-47e8-b4ab-39e19f016e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:08.293562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff57e7c-a471-47e8-b4ab-39e19f016e9d service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:08.293896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff57e7c-a471-47e8-b4ab-39e19f016e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:08.294174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff57e7c-a471-47e8-b4ab-39e19f016e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:08.301158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4ff57e7c-a471-47e8-b4ab-39e19f016e9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:08.312485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f89f4-1bad-4f00-abc6-2b5b0e4017e7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:08.333071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d261902-217a-4301-81c6-54ef15cf8035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:08.333853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d261902-217a-4301-81c6-54ef15cf8035 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:08.334235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d261902-217a-4301-81c6-54ef15cf8035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:08.336296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d261902-217a-4301-81c6-54ef15cf8035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:08.337010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d261902-217a-4301-81c6-54ef15cf8035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:08.339260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d261902-217a-4301-81c6-54ef15cf8035 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:08.374608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 846/1697] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:08.380633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d261902-217a-4301-81c6-54ef15cf8035 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:08.408683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d261902-217a-4301-81c6-54ef15cf8035 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:08.433549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156928d-7127-40cb-99d1-1bdc95c03d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:08.434362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156928d-7127-40cb-99d1-1bdc95c03d90 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:08.434754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156928d-7127-40cb-99d1-1bdc95c03d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:08.435257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156928d-7127-40cb-99d1-1bdc95c03d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:08.435902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156928d-7127-40cb-99d1-1bdc95c03d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:08.438033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156928d-7127-40cb-99d1-1bdc95c03d90 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:08.481637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 847/1698] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:08.498895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156928d-7127-40cb-99d1-1bdc95c03d90 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:08.515805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff57e7c-a471-47e8-b4ab-39e19f016e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:08.522338 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff57e7c-a471-47e8-b4ab-39e19f016e9d service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:08.531428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156928d-7127-40cb-99d1-1bdc95c03d90 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:08.577372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:08.582352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:08.583256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:08.583482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 852/1699] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:08.583976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] PATH_INFO: `/v3/projects/9e9fa6fe84f547158ac0190fcb386663/users/6c43afa2ed154b27bb186d4aa1912f82/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:08.643131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9fa6fe84f547158ac0190fcb386663, user_id=6c43afa2ed154b27bb186d4aa1912f82, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:08.648009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:08.672829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 848/1700] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:08 2022] PUT /identity/v3/projects/9e9fa6fe84f547158ac0190fcb386663/users/6c43afa2ed154b27bb186d4aa1912f82/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:08.683225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c8d3c-6c42-4929-8ed1-ac766ff64663 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:08.683810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c8d3c-6c42-4929-8ed1-ac766ff64663 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:08.684413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c8d3c-6c42-4929-8ed1-ac766ff64663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:08.727834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ad8c8d3c-6c42-4929-8ed1-ac766ff64663 None None] MFA Rules not processed for user `6c43afa2ed154b27bb186d4aa1912f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:08.729088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ad8c8d3c-6c42-4929-8ed1-ac766ff64663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:08.868174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 853/1701] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:08.881853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:08.884178 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:09.229621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f8fae-5b29-40fc-a1ed-4d8c95ce5051 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:09.237900 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ce4b2-2873-4189-9151-c967707cc4dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:09.242672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ce4b2-2873-4189-9151-c967707cc4dd service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:09.242960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ce4b2-2873-4189-9151-c967707cc4dd service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:09.243863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ce4b2-2873-4189-9151-c967707cc4dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:09.280998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ce4b2-2873-4189-9151-c967707cc4dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:09.282626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ce4b2-2873-4189-9151-c967707cc4dd service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:09.310268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 849/1702] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:10.079317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b365f3-25a7-4c89-b6e5-10029d86a74b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:10.080187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b365f3-25a7-4c89-b6e5-10029d86a74b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:10.080611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b365f3-25a7-4c89-b6e5-10029d86a74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:10.125988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-95b365f3-25a7-4c89-b6e5-10029d86a74b None None] MFA Rules not processed for user `3a5ab962b58f473f874bd94bfe2db6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:10.127258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-95b365f3-25a7-4c89-b6e5-10029d86a74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:10.225276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 854/1703] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:12.419637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b1f0-4b5b-4aab-ad79-f1602ae0b437 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:12.420761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b1f0-4b5b-4aab-ad79-f1602ae0b437 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:12.421216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b1f0-4b5b-4aab-ad79-f1602ae0b437 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:12.425364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 850/1704] 10.222.0.43 () {54 vars in 912 bytes} [Thu Aug 11 16:28:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:28:12.441395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635785b-f0c5-42b2-a91a-9bfbf6145db3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:12.441855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635785b-f0c5-42b2-a91a-9bfbf6145db3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:12.441929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635785b-f0c5-42b2-a91a-9bfbf6145db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:12.454740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:12.559934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8635785b-f0c5-42b2-a91a-9bfbf6145db3 None None] MFA Rules not processed for user `123d33bb6caa464aba4f9be6b0fb7656`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:12.561371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8635785b-f0c5-42b2-a91a-9bfbf6145db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:12.718064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 855/1705] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:28:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:12.923103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b1f0-4b5b-4aab-ad79-f1602ae0b437 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:12.925017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-df34b1f0-4b5b-4aab-ad79-f1602ae0b437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:12.956811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b1f0-4b5b-4aab-ad79-f1602ae0b437 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:12.971839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cc7a8-4941-4a79-ab16-6abb35f48258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:12.972511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc7a8-4941-4a79-ab16-6abb35f48258 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:12.972793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc7a8-4941-4a79-ab16-6abb35f48258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:12.973065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc7a8-4941-4a79-ab16-6abb35f48258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:13.004476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6cc7a8-4941-4a79-ab16-6abb35f48258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:13.011650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6cc7a8-4941-4a79-ab16-6abb35f48258 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:13.050177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 851/1706] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:14.060033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635785b-f0c5-42b2-a91a-9bfbf6145db3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:14.277060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affebd2c-1d54-4cde-a94a-3eb6e2eeca6b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:14.280882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affebd2c-1d54-4cde-a94a-3eb6e2eeca6b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:14.285764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affebd2c-1d54-4cde-a94a-3eb6e2eeca6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:14.299785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635785b-f0c5-42b2-a91a-9bfbf6145db3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:14.329625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:14.330293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:14.330569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:14.331007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] PATH_INFO: `/v3/users/6c43afa2ed154b27bb186d4aa1912f82` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:14.332852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c43afa2ed154b27bb186d4aa1912f82)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:14.336076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:14.402297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-affebd2c-1d54-4cde-a94a-3eb6e2eeca6b None None] MFA Rules not processed for user `60ee85a52c0244879f10e8943d20dfb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:14.403830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-affebd2c-1d54-4cde-a94a-3eb6e2eeca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:14.454469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c43afa2ed154b27bb186d4aa1912f82'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:14.470347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c43afa2ed154b27bb186d4aa1912f82'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:14.471570 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c43afa2ed154b27bb186d4aa1912f82'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:14.510619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c43afa2ed154b27bb186d4aa1912f82'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:14.513869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3dcf5e2e-99ab-4d40-82b9-4312e1d813ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c43afa2ed154b27bb186d4aa1912f82'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:14.554511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 856/1707] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:14 2022] DELETE /identity/v3/users/6c43afa2ed154b27bb186d4aa1912f82 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:14.560310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae767667-5123-43aa-b6ea-afb94f7a0d26 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:14.563006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae767667-5123-43aa-b6ea-afb94f7a0d26 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:14.564040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae767667-5123-43aa-b6ea-afb94f7a0d26 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:14.588861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 857/1708] 10.222.0.43 () {54 vars in 961 bytes} [Thu Aug 11 16:28:14 2022] GET /identity => generated 270 bytes in 33 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:28:14.594200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be114-220d-4059-9781-04841c42a6f8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:14.594494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be114-220d-4059-9781-04841c42a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:14.594770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3be114-220d-4059-9781-04841c42a6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:14.646529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:14.650257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ba3be114-220d-4059-9781-04841c42a6f8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:14.654853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ba3be114-220d-4059-9781-04841c42a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:14.682836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 852/1709] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:14.682836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfdc6b-d800-4844-bff6-999f5139c8b7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:14.682836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfdc6b-d800-4844-bff6-999f5139c8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:14.682836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfdc6b-d800-4844-bff6-999f5139c8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:14.696356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:14.728835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-91bfdc6b-d800-4844-bff6-999f5139c8b7 None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:14.736832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-91bfdc6b-d800-4844-bff6-999f5139c8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:14.835310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 858/1710] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:14.837333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3be114-220d-4059-9781-04841c42a6f8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:14.841059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ba3be114-220d-4059-9781-04841c42a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:14.915311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 853/1711] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Aug 11 16:28:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:14.917777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd057a-f3ac-498c-b270-81304f6f443a None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:14.918068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd057a-f3ac-498c-b270-81304f6f443a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:14.918339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd057a-f3ac-498c-b270-81304f6f443a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:14.987669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-2bfd057a-f3ac-498c-b270-81304f6f443a None None] MFA Rules not processed for user `5d59680b3fde47bf81c7765ca4d8a757`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:14.990452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2bfd057a-f3ac-498c-b270-81304f6f443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:15.043868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3be114-220d-4059-9781-04841c42a6f8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:15.066365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4806b2-e8f6-41f3-873b-affa4cdeff2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:15.069069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4806b2-e8f6-41f3-873b-affa4cdeff2f service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:15.070764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4806b2-e8f6-41f3-873b-affa4cdeff2f service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:15.071068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4806b2-e8f6-41f3-873b-affa4cdeff2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:15.149775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 854/1712] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:15.149775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd057a-f3ac-498c-b270-81304f6f443a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:15.149775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2bfd057a-f3ac-498c-b270-81304f6f443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:15.287201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd057a-f3ac-498c-b270-81304f6f443a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:15.294167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e81b42-666c-4f84-9d10-a959c9020b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:15.294901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e81b42-666c-4f84-9d10-a959c9020b58 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:15.295373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e81b42-666c-4f84-9d10-a959c9020b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:15.295805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e81b42-666c-4f84-9d10-a959c9020b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:15.298478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4806b2-e8f6-41f3-873b-affa4cdeff2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:15.300031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4806b2-e8f6-41f3-873b-affa4cdeff2f service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:15.309538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e81b42-666c-4f84-9d10-a959c9020b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:15.312538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e81b42-666c-4f84-9d10-a959c9020b58 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:15.325898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 855/1713] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:15.327045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e81b42-666c-4f84-9d10-a959c9020b58 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:15.330250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 859/1714] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:15.331647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996faa6e-51fd-46bf-9ba4-948b1e297b59 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:15.332080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996faa6e-51fd-46bf-9ba4-948b1e297b59 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:15.332482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996faa6e-51fd-46bf-9ba4-948b1e297b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:15.373937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e81b42-666c-4f84-9d10-a959c9020b58 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:15.386861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:15.388557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:15.390570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:15.391071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:15.393868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-996faa6e-51fd-46bf-9ba4-948b1e297b59 None None] MFA Rules not processed for user `5d59680b3fde47bf81c7765ca4d8a757`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:15.396281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-996faa6e-51fd-46bf-9ba4-948b1e297b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:15.399272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:15.416409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 860/1715] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:15.423015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996faa6e-51fd-46bf-9ba4-948b1e297b59 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:15.426834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-996faa6e-51fd-46bf-9ba4-948b1e297b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:15.434237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:15.436350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:15.449987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 856/1716] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:15.451427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:15.453054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:15.472463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae59c7d-2ef9-4b9a-a32f-4ec5dfced745 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:15.472969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996faa6e-51fd-46bf-9ba4-948b1e297b59 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:15.481390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926c863-1b90-49a0-966b-c7aa95fba7a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:15.482251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c863-1b90-49a0-966b-c7aa95fba7a6 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:15.482752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c863-1b90-49a0-966b-c7aa95fba7a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:15.483303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c863-1b90-49a0-966b-c7aa95fba7a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:15.491567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:15.492415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:15.494191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:15.494498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:15.494782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] PATH_INFO: `/v3/users/6aa18ff44d5e4efda91a7fe889a456bc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:15.498347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926c863-1b90-49a0-966b-c7aa95fba7a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:15.500343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926c863-1b90-49a0-966b-c7aa95fba7a6 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:15.501897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa18ff44d5e4efda91a7fe889a456bc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:15.503632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:15.525771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 857/1717] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:15.642632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:15.657128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa18ff44d5e4efda91a7fe889a456bc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:15.684016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa18ff44d5e4efda91a7fe889a456bc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:15.684929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa18ff44d5e4efda91a7fe889a456bc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:15.745279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa18ff44d5e4efda91a7fe889a456bc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:15.747904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa18ff44d5e4efda91a7fe889a456bc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:15.805112 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 861/1718] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:15 2022] DELETE /identity/v3/users/6aa18ff44d5e4efda91a7fe889a456bc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:15.810287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea61d-d53b-44c2-9a78-a925f689200c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:15.810574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea61d-d53b-44c2-9a78-a925f689200c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:15.810900 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aea61d-d53b-44c2-9a78-a925f689200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:15.863430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c6aea61d-d53b-44c2-9a78-a925f689200c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:15.866228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c6aea61d-d53b-44c2-9a78-a925f689200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:16.032264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 858/1719] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:16.097506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:16.100637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:16.213729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea61d-d53b-44c2-9a78-a925f689200c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:16.214214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae449e-9145-4f0f-a8c1-da2161471561 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:16.215417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c6aea61d-d53b-44c2-9a78-a925f689200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:16.223899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be09430-1ab6-4159-a368-26f884233fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:16.225492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be09430-1ab6-4159-a368-26f884233fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:16.228077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be09430-1ab6-4159-a368-26f884233fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:16.228671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be09430-1ab6-4159-a368-26f884233fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:16.243455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aea61d-d53b-44c2-9a78-a925f689200c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:16.244998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be09430-1ab6-4159-a368-26f884233fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:16.248712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be09430-1ab6-4159-a368-26f884233fb0 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:16.258444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:16.259092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:16.259536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:16.259818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] PATH_INFO: `/v3/users/bfc5b9cd2d0045c6a0113c5e7e3c6e78` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:16.261034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc5b9cd2d0045c6a0113c5e7e3c6e78)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:16.276994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:16.286993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 862/1720] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:16.374943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc5b9cd2d0045c6a0113c5e7e3c6e78'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:16.401016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc5b9cd2d0045c6a0113c5e7e3c6e78'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:16.402194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc5b9cd2d0045c6a0113c5e7e3c6e78'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:16.417139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc5b9cd2d0045c6a0113c5e7e3c6e78'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:16.428135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc5b9cd2d0045c6a0113c5e7e3c6e78'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:16.436827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 859/1721] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:16 2022] DELETE /identity/v3/users/bfc5b9cd2d0045c6a0113c5e7e3c6e78 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:16.440994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97d4c-08fe-4fca-a2e2-91da8a43c3c9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:16.441356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97d4c-08fe-4fca-a2e2-91da8a43c3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:16.441590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded97d4c-08fe-4fca-a2e2-91da8a43c3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:16.466392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ded97d4c-08fe-4fca-a2e2-91da8a43c3c9 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:16.467557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ded97d4c-08fe-4fca-a2e2-91da8a43c3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:16.599120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 863/1722] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:16.682456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:16.684659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:16.819050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311923c2-22fd-4805-9ac8-a4a87c3f34ae admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:16.824660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05a9af-465b-433e-acab-9f015256c5ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:16.825299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a9af-465b-433e-acab-9f015256c5ab service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:16.825580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a9af-465b-433e-acab-9f015256c5ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:16.825848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05a9af-465b-433e-acab-9f015256c5ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:16.849968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05a9af-465b-433e-acab-9f015256c5ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:16.851953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05a9af-465b-433e-acab-9f015256c5ab service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:16.877246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 860/1723] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:17.207764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe697-fddd-40fa-9402-8a8891369d9a None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:17.208316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe697-fddd-40fa-9402-8a8891369d9a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:17.208697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fe697-fddd-40fa-9402-8a8891369d9a None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:17.216918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 864/1724] 10.222.0.43 () {54 vars in 912 bytes} [Thu Aug 11 16:28:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:28:17.243567 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b78f8-d3fa-4292-a139-61480509bbaf None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:17.243864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b78f8-d3fa-4292-a139-61480509bbaf None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:17.244529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b78f8-d3fa-4292-a139-61480509bbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:17.300381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f17b78f8-d3fa-4292-a139-61480509bbaf None None] MFA Rules not processed for user `123d33bb6caa464aba4f9be6b0fb7656`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:17.302295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f17b78f8-d3fa-4292-a139-61480509bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:17.327889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e588e1e-64d8-434d-bee3-e49213dbeccd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:17.328394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e588e1e-64d8-434d-bee3-e49213dbeccd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:17.329277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e588e1e-64d8-434d-bee3-e49213dbeccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:17.348975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 861/1725] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:17.403346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b78f8-d3fa-4292-a139-61480509bbaf None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:17.409111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f17b78f8-d3fa-4292-a139-61480509bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:17.426845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:17.429711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5e588e1e-64d8-434d-bee3-e49213dbeccd None None] MFA Rules not processed for user `e7e5b04101c1403d9b3a311803ac798f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:17.437217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5e588e1e-64d8-434d-bee3-e49213dbeccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:17.458011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b78f8-d3fa-4292-a139-61480509bbaf None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:17.480160 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:17.481012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:17.481379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:17.481763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] PATH_INFO: `/v3/projects/9e9fa6fe84f547158ac0190fcb386663` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:17.484504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9fa6fe84f547158ac0190fcb386663)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:17.486602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:17.655515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9fa6fe84f547158ac0190fcb386663', 'request_id': 'req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:17.657747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:17.671807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9fa6fe84f547158ac0190fcb386663', 'request_id': 'req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:17.684385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 865/1726] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:17.685899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e588e1e-64d8-434d-bee3-e49213dbeccd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:17.710130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9fa6fe84f547158ac0190fcb386663', 'request_id': 'req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:17.721800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e588e1e-64d8-434d-bee3-e49213dbeccd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:17.734077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] The token cache is being invalidate because project 9e9fa6fe84f547158ac0190fcb386663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:17.734594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:17.736092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ee60ab5-a9f2-4966-8f00-27eaf912ad83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:17.741410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 862/1727] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:17 2022] DELETE /identity/v3/projects/9e9fa6fe84f547158ac0190fcb386663 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:17.747028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504c1c-a805-4cfc-be5b-38bbe8bb0537 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:17.747521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504c1c-a805-4cfc-be5b-38bbe8bb0537 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:17.748129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504c1c-a805-4cfc-be5b-38bbe8bb0537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:17.825216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-eb504c1c-a805-4cfc-be5b-38bbe8bb0537 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:17.826456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-eb504c1c-a805-4cfc-be5b-38bbe8bb0537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:17.865840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:17.867404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:17.867937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:17.868513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:17.876133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:17.890333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:17.892010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:17.903679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 866/1728] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:17.905011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:17.907114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:17.953097 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 863/1729] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:17.967971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a386d-174a-48ca-8ead-648a4601323f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:17.967971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a386d-174a-48ca-8ead-648a4601323f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:17.968255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a386d-174a-48ca-8ead-648a4601323f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.014675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-244a386d-174a-48ca-8ead-648a4601323f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:18.015928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-244a386d-174a-48ca-8ead-648a4601323f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:18.027294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 864/1730] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:18.036033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a386d-174a-48ca-8ead-648a4601323f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:18.068129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a386d-174a-48ca-8ead-648a4601323f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:18.086667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:18.087302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.087577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.087846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.088540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:18.089438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:18.091132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:18.093492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33cc90-c24c-4377-92ec-43a432446a59 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:18.110238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:18.111047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.111353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.111616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.115762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:18.130612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 865/1731] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:28:18.131812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:18.151065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21611b50-c96b-4c5a-8b5f-dc5b00063164 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:18.165743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dd37a-9613-42c9-bbb5-edf368ff429d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:18.166371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd37a-9613-42c9-bbb5-edf368ff429d admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.166649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd37a-9613-42c9-bbb5-edf368ff429d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.166915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dd37a-9613-42c9-bbb5-edf368ff429d admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.167562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dd37a-9613-42c9-bbb5-edf368ff429d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:18.169542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dd37a-9613-42c9-bbb5-edf368ff429d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:18.209389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:18.229648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:18.239960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 866/1732] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:18 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:18.241059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dd37a-9613-42c9-bbb5-edf368ff429d admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:18.249115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 867/1733] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:18.373583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:18.417340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dd37a-9613-42c9-bbb5-edf368ff429d admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:18.437410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dc0af-e012-429b-9580-a46289da318b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:18.437859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc0af-e012-429b-9580-a46289da318b admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.438319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc0af-e012-429b-9580-a46289da318b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.438836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861dc0af-e012-429b-9580-a46289da318b admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.439699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dc0af-e012-429b-9580-a46289da318b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:18.441891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861dc0af-e012-429b-9580-a46289da318b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:18.480061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:18.489135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 867/1734] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:18 2022] POST /identity/v3/users => generated 303 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:18.499406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b573cfc4-e932-4dd3-b99d-583824ccd0b6 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:18.512961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cd072-37c7-4878-aa0d-0a74b4ca6d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:18.514008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cd072-37c7-4878-aa0d-0a74b4ca6d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.514008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cd072-37c7-4878-aa0d-0a74b4ca6d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.514190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cd072-37c7-4878-aa0d-0a74b4ca6d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.514777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cd072-37c7-4878-aa0d-0a74b4ca6d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:18.516425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cd072-37c7-4878-aa0d-0a74b4ca6d73 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:18.542943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 868/1735] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:18.547348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dc0af-e012-429b-9580-a46289da318b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:18.561981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861dc0af-e012-429b-9580-a46289da318b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:18.576414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:18.577043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.577419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.577871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.578695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:18.580656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:18.638228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 868/1736] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:18.642403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cd072-37c7-4878-aa0d-0a74b4ca6d73 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:18.662194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:18.665436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:18.669468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cd072-37c7-4878-aa0d-0a74b4ca6d73 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:18.685562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcfcbb-fbb3-4ba9-832a-eba77c912608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:18.686248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfcbb-fbb3-4ba9-832a-eba77c912608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.686602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfcbb-fbb3-4ba9-832a-eba77c912608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.686892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcfcbb-fbb3-4ba9-832a-eba77c912608 admin admin] PATH_INFO: `/v3/projects/afb173ec8f4b42349b2c3dbc4de6444a/users/9583c78a3d1241fdb7e22ff2d0a5a2d2/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.693511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083cfb9-b541-46b4-a66f-daf2b19859d1 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:18.707176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:18.707830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.708125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.708404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] PATH_INFO: `/v3/projects/b70ab251b5d64351a2fe0cf171ef6f26` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.709578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b70ab251b5d64351a2fe0cf171ef6f26)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:18.712538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:18.736817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcfcbb-fbb3-4ba9-832a-eba77c912608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb173ec8f4b42349b2c3dbc4de6444a, user_id=9583c78a3d1241fdb7e22ff2d0a5a2d2, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:18.739509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcfcbb-fbb3-4ba9-832a-eba77c912608 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:18.759383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:18.789007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 869/1737] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:18 2022] PUT /identity/v3/projects/afb173ec8f4b42349b2c3dbc4de6444a/users/9583c78a3d1241fdb7e22ff2d0a5a2d2/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:18.790708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32c9d-7b7a-4df4-9249-8f14ebd581ea None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.790993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32c9d-7b7a-4df4-9249-8f14ebd581ea None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.791266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e32c9d-7b7a-4df4-9249-8f14ebd581ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:18.866747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-99e32c9d-7b7a-4df4-9249-8f14ebd581ea None None] MFA Rules not processed for user `eca40eca0b6140a498e034e3e37a8c47`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:18.868905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99e32c9d-7b7a-4df4-9249-8f14ebd581ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:18.871200 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b70ab251b5d64351a2fe0cf171ef6f26', 'request_id': 'req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:18.885564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b70ab251b5d64351a2fe0cf171ef6f26', 'request_id': 'req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:18.906049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b70ab251b5d64351a2fe0cf171ef6f26', 'request_id': 'req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:18.950349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] The token cache is being invalidate because project b70ab251b5d64351a2fe0cf171ef6f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:18.950646 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:18.955635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce3d00ca-a71b-49dc-a233-54f1f3097aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:18.964217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 869/1738] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:18 2022] DELETE /identity/v3/projects/b70ab251b5d64351a2fe0cf171ef6f26 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:18.965808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbfef9-e966-4d5d-9078-060f421a7601 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:18.966096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbfef9-e966-4d5d-9078-060f421a7601 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:18.966367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbfef9-e966-4d5d-9078-060f421a7601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:19.049368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5fbbfef9-e966-4d5d-9078-060f421a7601 None None] MFA Rules not processed for user `9583c78a3d1241fdb7e22ff2d0a5a2d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:19.050946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5fbbfef9-e966-4d5d-9078-060f421a7601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:19.084810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 870/1739] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:19.110366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e32c9d-7b7a-4df4-9249-8f14ebd581ea None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:19.111985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99e32c9d-7b7a-4df4-9249-8f14ebd581ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:19.261789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 870/1740] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:18 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:19.267932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac169f9a-ae4e-461c-bc39-c33dd2707a83 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:19.269650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac169f9a-ae4e-461c-bc39-c33dd2707a83 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:19.269954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac169f9a-ae4e-461c-bc39-c33dd2707a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:19.340074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e32c9d-7b7a-4df4-9249-8f14ebd581ea None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:19.348324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:19.349560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:19.357060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:19.358064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] PATH_INFO: `/v3/users/215e9785968f41668aaf20d95ef4a9bd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:19.360324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215e9785968f41668aaf20d95ef4a9bd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:19.362796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:19.433660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ac169f9a-ae4e-461c-bc39-c33dd2707a83 None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:19.435113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ac169f9a-ae4e-461c-bc39-c33dd2707a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:19.484577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:19.506549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215e9785968f41668aaf20d95ef4a9bd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.525198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215e9785968f41668aaf20d95ef4a9bd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.526531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215e9785968f41668aaf20d95ef4a9bd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.563114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215e9785968f41668aaf20d95ef4a9bd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.567386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-65b4aa65-cb44-4d58-81ab-5844d4e5f970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215e9785968f41668aaf20d95ef4a9bd'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.597698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 871/1741] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:19 2022] DELETE /identity/v3/users/215e9785968f41668aaf20d95ef4a9bd => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:19.599350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850490a-af02-49c3-ba55-d19f010db72d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:19.599873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850490a-af02-49c3-ba55-d19f010db72d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:19.599979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850490a-af02-49c3-ba55-d19f010db72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:19.619494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 871/1742] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:28:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:19.620684 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac169f9a-ae4e-461c-bc39-c33dd2707a83 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:19.624126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ac169f9a-ae4e-461c-bc39-c33dd2707a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:19.633741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:19.644770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f850490a-af02-49c3-ba55-d19f010db72d None None] MFA Rules not processed for user `eca40eca0b6140a498e034e3e37a8c47`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:19.653264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f850490a-af02-49c3-ba55-d19f010db72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:19.653444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac169f9a-ae4e-461c-bc39-c33dd2707a83 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:19.772042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 872/1743] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:19.772042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850490a-af02-49c3-ba55-d19f010db72d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:19.782187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f850490a-af02-49c3-ba55-d19f010db72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:19.782327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b849189-404c-443d-8266-f4c57a72d72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:19.782952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849189-404c-443d-8266-f4c57a72d72a admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:19.783114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849189-404c-443d-8266-f4c57a72d72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:19.783519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b849189-404c-443d-8266-f4c57a72d72a admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:19.784154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b849189-404c-443d-8266-f4c57a72d72a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:19.786512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b849189-404c-443d-8266-f4c57a72d72a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:19.806038 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850490a-af02-49c3-ba55-d19f010db72d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:19.811994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:19.812922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:19.813354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:19.813730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] PATH_INFO: `/v3/projects/a28ea2427c44437cae29bfe3dd4a2c82` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:19.815005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28ea2427c44437cae29bfe3dd4a2c82)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:19.817609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:19.848579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 872/1744] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:19 2022] POST /identity/v3/projects => generated 405 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:19.850230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6800d7c-5685-464f-896a-997e57fe6940 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:19.852501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6800d7c-5685-464f-896a-997e57fe6940 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:19.856265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6800d7c-5685-464f-896a-997e57fe6940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:19.888512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e6800d7c-5685-464f-896a-997e57fe6940 None None] MFA Rules not processed for user `5d59680b3fde47bf81c7765ca4d8a757`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:19.889926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e6800d7c-5685-464f-896a-997e57fe6940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:19.929756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28ea2427c44437cae29bfe3dd4a2c82', 'request_id': 'req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.948767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28ea2427c44437cae29bfe3dd4a2c82', 'request_id': 'req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.962252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28ea2427c44437cae29bfe3dd4a2c82', 'request_id': 'req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.984130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] The token cache is being invalidate because project a28ea2427c44437cae29bfe3dd4a2c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:19.984806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.986141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:19.990260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 873/1745] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:19 2022] DELETE /identity/v3/projects/a28ea2427c44437cae29bfe3dd4a2c82 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:19.994180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:19.996138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:20.056078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 873/1746] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:20.057588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6800d7c-5685-464f-896a-997e57fe6940 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:20.061741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e6800d7c-5685-464f-896a-997e57fe6940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:20.161786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:20.204950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6800d7c-5685-464f-896a-997e57fe6940 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:20.246841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ffa12-6bf9-4ee3-9981-0980bb89cc21 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:20.254838 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:20.255866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:20.256472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:20.257345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:20.263441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:20.326576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:20.327450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:20.327876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:20.328318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:20.329114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:20.338564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:20.395850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 874/1747] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:20 2022] POST /identity/v3/users => generated 309 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:20.396985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:20.401518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:20.406760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:20.512109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:20.514328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:20.533189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 874/1748] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:20.536312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:20.538290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:20.578362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a1bc4-4222-435b-b5b4-d0161ba6073f service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:20.590800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:20.591703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:20.593402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:20.595260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] PATH_INFO: `/v3/users/1ddbb2a95c30448193a5e746a68fd59e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:20.597290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ddbb2a95c30448193a5e746a68fd59e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:20.605544 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:20.765717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982cdcb-ed51-49c1-9b61-76f77b497729 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:20.782270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:20.783433 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:20.783935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:20.784486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:20.789466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:20.814374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ddbb2a95c30448193a5e746a68fd59e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:20.833041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ddbb2a95c30448193a5e746a68fd59e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:20.833920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ddbb2a95c30448193a5e746a68fd59e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:20.873266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ddbb2a95c30448193a5e746a68fd59e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:20.874402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ddbb2a95c30448193a5e746a68fd59e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:20.908131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 875/1749] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:20 2022] DELETE /identity/v3/users/1ddbb2a95c30448193a5e746a68fd59e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:20.915532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:20.917205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:20.946152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:20.948092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:20.958080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:20.971255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 875/1750] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:28:21.193735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e4c6-24cf-46f4-ba20-d9ebfdff63c7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:21.213057 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75db7-e97f-4a1f-88a1-91cf10e6c814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:21.213803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75db7-e97f-4a1f-88a1-91cf10e6c814 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:21.214157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75db7-e97f-4a1f-88a1-91cf10e6c814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:21.220628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc75db7-e97f-4a1f-88a1-91cf10e6c814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:21.221532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc75db7-e97f-4a1f-88a1-91cf10e6c814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:21.223521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc75db7-e97f-4a1f-88a1-91cf10e6c814 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:21.265841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 876/1751] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:21.270355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:21.299243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0843852-eb3d-4aee-aa20-9e16d3f73387 service swift] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:21.329503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adf318-0d24-4019-86d1-814eb72ccc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:21.332323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adf318-0d24-4019-86d1-814eb72ccc93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:21.339724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adf318-0d24-4019-86d1-814eb72ccc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:21.341449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adf318-0d24-4019-86d1-814eb72ccc93 admin admin] PATH_INFO: `/v3/projects/796008fb76e34a5a9b0b207a91c8d1a2/users/a2cebb42d3294de096c84764cbfa272e/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:21.425194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adf318-0d24-4019-86d1-814eb72ccc93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796008fb76e34a5a9b0b207a91c8d1a2, user_id=a2cebb42d3294de096c84764cbfa272e, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:21.443477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adf318-0d24-4019-86d1-814eb72ccc93 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:21.509762 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 876/1752] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:21 2022] PUT /identity/v3/projects/796008fb76e34a5a9b0b207a91c8d1a2/users/a2cebb42d3294de096c84764cbfa272e/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:21.511332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75db7-e97f-4a1f-88a1-91cf10e6c814 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:21.543861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc75db7-e97f-4a1f-88a1-91cf10e6c814 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:21.660068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:21.768976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d2a21-9bcc-4c25-9a8b-46fd2461b6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:21.769887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d2a21-9bcc-4c25-9a8b-46fd2461b6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:21.770261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d2a21-9bcc-4c25-9a8b-46fd2461b6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:21.770889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d2a21-9bcc-4c25-9a8b-46fd2461b6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:21.771341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d2a21-9bcc-4c25-9a8b-46fd2461b6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:21.773827 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d2a21-9bcc-4c25-9a8b-46fd2461b6bf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:21.825011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 877/1753] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:21.826154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adf318-0d24-4019-86d1-814eb72ccc93 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:21.844350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adf318-0d24-4019-86d1-814eb72ccc93 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:21.869019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:21.871329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:21.871673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:21.871885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] PATH_INFO: `/v3/projects/796008fb76e34a5a9b0b207a91c8d1a2/users/a2cebb42d3294de096c84764cbfa272e/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:21.881127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796008fb76e34a5a9b0b207a91c8d1a2, user_id=a2cebb42d3294de096c84764cbfa272e, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:21.887886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:21.935824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 877/1754] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:21 2022] PUT /identity/v3/projects/796008fb76e34a5a9b0b207a91c8d1a2/users/a2cebb42d3294de096c84764cbfa272e/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:21.947872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2f759-432f-4de5-a622-4b67cdf31be1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:21.948472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2f759-432f-4de5-a622-4b67cdf31be1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:21.948939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2f759-432f-4de5-a622-4b67cdf31be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:22.033629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-aaa2f759-432f-4de5-a622-4b67cdf31be1 None None] MFA Rules not processed for user `a2cebb42d3294de096c84764cbfa272e`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:22.035247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-aaa2f759-432f-4de5-a622-4b67cdf31be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:22.200774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 878/1755] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:21 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:22.209819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:22.215445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:22.329819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591bd6a-71ee-4c99-ace7-99fe02144122 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:22.339430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:22.340327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:22.340775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:22.341196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:22.353590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:22.355365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:22.386562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 878/1756] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:22 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:22.793964 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2f759-432f-4de5-a622-4b67cdf31be1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:22.965075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:22.966712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2f759-432f-4de5-a622-4b67cdf31be1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:22.968497 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:22.988572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:22.988572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:22.988572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:22.988572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] PATH_INFO: `/v3/users/9583c78a3d1241fdb7e22ff2d0a5a2d2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:22.989547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9583c78a3d1241fdb7e22ff2d0a5a2d2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:22.989547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:23.009060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a7211-ac39-46dd-8178-b0d84854ae4b service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:23.030540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:23.031197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:23.031475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:23.031753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] PATH_INFO: `/v3/projects/58a700e918a84de49b7002d1d2530779` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:23.035420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58a700e918a84de49b7002d1d2530779)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:23.047420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:23.152607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9583c78a3d1241fdb7e22ff2d0a5a2d2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.168621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9583c78a3d1241fdb7e22ff2d0a5a2d2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.174276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9583c78a3d1241fdb7e22ff2d0a5a2d2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.200747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58a700e918a84de49b7002d1d2530779', 'request_id': 'req-f079b77a-dbfc-439d-bcab-8950bf695349'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.206961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:23.217882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58a700e918a84de49b7002d1d2530779', 'request_id': 'req-f079b77a-dbfc-439d-bcab-8950bf695349'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.229643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9583c78a3d1241fdb7e22ff2d0a5a2d2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.230495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2dbc0324-58e6-4f78-8e8a-a542676fdfd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9583c78a3d1241fdb7e22ff2d0a5a2d2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.236025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58a700e918a84de49b7002d1d2530779', 'request_id': 'req-f079b77a-dbfc-439d-bcab-8950bf695349'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.260080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] The token cache is being invalidate because project 58a700e918a84de49b7002d1d2530779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:23.260080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.261281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f079b77a-dbfc-439d-bcab-8950bf695349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:23.267314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 879/1757] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:22 2022] DELETE /identity/v3/projects/58a700e918a84de49b7002d1d2530779 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:23.269296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264d15-57c4-43bd-ab61-0a46acb2c4d4 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:23.271613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264d15-57c4-43bd-ab61-0a46acb2c4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:23.271909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264d15-57c4-43bd-ab61-0a46acb2c4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:23.278114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 879/1758] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:22 2022] DELETE /identity/v3/users/9583c78a3d1241fdb7e22ff2d0a5a2d2 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:23.280306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c36d-57f1-4a30-98d4-6200f80ab839 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:23.280606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c36d-57f1-4a30-98d4-6200f80ab839 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:23.281147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c36d-57f1-4a30-98d4-6200f80ab839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:23.330300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-70b9c36d-57f1-4a30-98d4-6200f80ab839 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:23.332248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-70b9c36d-57f1-4a30-98d4-6200f80ab839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:23.333475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-05264d15-57c4-43bd-ab61-0a46acb2c4d4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:23.336298 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-05264d15-57c4-43bd-ab61-0a46acb2c4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:23.489514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 880/1759] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:23.516308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:23.532280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9c36d-57f1-4a30-98d4-6200f80ab839 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:23.534298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-70b9c36d-57f1-4a30-98d4-6200f80ab839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:23.633095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 880/1760] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:23.640701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327aa62-2f5f-48dd-8f22-88cf017c5e8b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:23.642276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327aa62-2f5f-48dd-8f22-88cf017c5e8b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:23.642822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327aa62-2f5f-48dd-8f22-88cf017c5e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:23.685250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-6327aa62-2f5f-48dd-8f22-88cf017c5e8b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:23.687559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6327aa62-2f5f-48dd-8f22-88cf017c5e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:23.705407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 881/1761] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:23.710347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327aa62-2f5f-48dd-8f22-88cf017c5e8b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:23.741897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327aa62-2f5f-48dd-8f22-88cf017c5e8b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:23.775007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:23.777324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:23.787037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:23.788910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:23.790231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:23.792556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:23.800170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:23.826472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9c36d-57f1-4a30-98d4-6200f80ab839 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:23.861257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 882/1762] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:28:23.864576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:23.874419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fe3b2-7d51-422b-b3bd-128270a43ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:23.875080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fe3b2-7d51-422b-b3bd-128270a43ecf service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:23.875321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fe3b2-7d51-422b-b3bd-128270a43ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:23.875639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fe3b2-7d51-422b-b3bd-128270a43ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:23.937242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede6eb8-9644-485e-ad42-0a797ae10e53 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:23.941579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fe3b2-7d51-422b-b3bd-128270a43ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:23.947108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fe3b2-7d51-422b-b3bd-128270a43ecf service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:23.949801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792fb6f-b1f1-4071-ad28-8b15ec3f1c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:23.950549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792fb6f-b1f1-4071-ad28-8b15ec3f1c7f admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:23.950910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792fb6f-b1f1-4071-ad28-8b15ec3f1c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:23.951697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792fb6f-b1f1-4071-ad28-8b15ec3f1c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:23.953483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792fb6f-b1f1-4071-ad28-8b15ec3f1c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:23.956499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792fb6f-b1f1-4071-ad28-8b15ec3f1c7f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:23.983533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 881/1763] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:24.037163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 883/1764] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:23 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:24.038521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fe3b2-7d51-422b-b3bd-128270a43ecf service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:24.055304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fe3b2-7d51-422b-b3bd-128270a43ecf service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:24.059293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a06c-48a8-440a-a100-6b9705fc2d94 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:24.060280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a06c-48a8-440a-a100-6b9705fc2d94 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:24.060280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a06c-48a8-440a-a100-6b9705fc2d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:24.139221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-39d4a06c-48a8-440a-a100-6b9705fc2d94 None None] MFA Rules not processed for user `123d33bb6caa464aba4f9be6b0fb7656`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:24.155279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-39d4a06c-48a8-440a-a100-6b9705fc2d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:24.252132 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe82d2e-4014-4280-8c1c-eadb3a91d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:24.252989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe82d2e-4014-4280-8c1c-eadb3a91d4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:24.253338 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe82d2e-4014-4280-8c1c-eadb3a91d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:24.253679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe82d2e-4014-4280-8c1c-eadb3a91d4f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:24.254391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe82d2e-4014-4280-8c1c-eadb3a91d4f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:24.260632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe82d2e-4014-4280-8c1c-eadb3a91d4f0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:24.298404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 882/1765] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:24 2022] POST /identity/v3/users => generated 288 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:24.303449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe82d2e-4014-4280-8c1c-eadb3a91d4f0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:24.327618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 884/1766] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Aug 11 16:28:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:24.337585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe82d2e-4014-4280-8c1c-eadb3a91d4f0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:24.345543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327b51-a2ce-4dab-8234-36cfa96ff126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:24.346317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327b51-a2ce-4dab-8234-36cfa96ff126 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:24.346655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327b51-a2ce-4dab-8234-36cfa96ff126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:24.347009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d327b51-a2ce-4dab-8234-36cfa96ff126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:24.348509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d327b51-a2ce-4dab-8234-36cfa96ff126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:24.350242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d327b51-a2ce-4dab-8234-36cfa96ff126 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:24.387407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4a06c-48a8-440a-a100-6b9705fc2d94 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:24.391375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 883/1767] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:24.409960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4a06c-48a8-440a-a100-6b9705fc2d94 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:24.415900 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49b64e-9846-419b-b91c-798212bb6912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:24.416770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49b64e-9846-419b-b91c-798212bb6912 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:24.417705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49b64e-9846-419b-b91c-798212bb6912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:24.418143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49b64e-9846-419b-b91c-798212bb6912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:24.419533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49b64e-9846-419b-b91c-798212bb6912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:24.421893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49b64e-9846-419b-b91c-798212bb6912 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:24.454046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327b51-a2ce-4dab-8234-36cfa96ff126 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:24.467277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:24.476001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 885/1768] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:24.482360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d327b51-a2ce-4dab-8234-36cfa96ff126 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:24.493896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df9e66-c235-4f08-8223-caf90e8e23bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:24.495106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df9e66-c235-4f08-8223-caf90e8e23bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:24.495651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df9e66-c235-4f08-8223-caf90e8e23bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:24.496390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df9e66-c235-4f08-8223-caf90e8e23bc admin admin] PATH_INFO: `/v3/projects/9b87a927ba4b4841b8e4db8c9dd6dbf4/users/2d7e0ddf97014038a53f33833acfc05f/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:24.518117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df9e66-c235-4f08-8223-caf90e8e23bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b87a927ba4b4841b8e4db8c9dd6dbf4, user_id=2d7e0ddf97014038a53f33833acfc05f, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:24.521406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df9e66-c235-4f08-8223-caf90e8e23bc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:24.557491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a1125-3e08-4e5c-b408-7b282c0d195f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:24.558039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a1125-3e08-4e5c-b408-7b282c0d195f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:24.558463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a1125-3e08-4e5c-b408-7b282c0d195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:24.562037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 884/1769] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:24 2022] PUT /identity/v3/projects/9b87a927ba4b4841b8e4db8c9dd6dbf4/users/2d7e0ddf97014038a53f33833acfc05f/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:24.647112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5d4a1125-3e08-4e5c-b408-7b282c0d195f None None] MFA Rules not processed for user `2d7e0ddf97014038a53f33833acfc05f`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:24.655028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5d4a1125-3e08-4e5c-b408-7b282c0d195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:24.876916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df9e66-c235-4f08-8223-caf90e8e23bc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:24.898273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 886/1770] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:25.005719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a01104-b5bc-4cb4-82aa-6031a397a5af None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:25.006232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a01104-b5bc-4cb4-82aa-6031a397a5af None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:25.007063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a01104-b5bc-4cb4-82aa-6031a397a5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:25.082010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-59a01104-b5bc-4cb4-82aa-6031a397a5af None None] MFA Rules not processed for user `60ee85a52c0244879f10e8943d20dfb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:25.090900 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-59a01104-b5bc-4cb4-82aa-6031a397a5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:25.230283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df9e66-c235-4f08-8223-caf90e8e23bc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:25.238518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:25.253550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8b402-fdc4-4f47-a826-a312a52b7bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:25.260476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b402-fdc4-4f47-a826-a312a52b7bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:25.263391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b402-fdc4-4f47-a826-a312a52b7bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:25.264007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8b402-fdc4-4f47-a826-a312a52b7bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:25.287725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8b402-fdc4-4f47-a826-a312a52b7bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:25.290007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8b402-fdc4-4f47-a826-a312a52b7bd0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:25.439967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 885/1771] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:24 2022] POST /identity/v3/projects => generated 377 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:25.439967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8b402-fdc4-4f47-a826-a312a52b7bd0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:25.476245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 887/1772] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:25.479207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a01104-b5bc-4cb4-82aa-6031a397a5af None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:25.547997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a01104-b5bc-4cb4-82aa-6031a397a5af None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:25.869941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d436774-c732-476f-a289-a698657cd83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:25.870586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d436774-c732-476f-a289-a698657cd83c admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:25.871022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d436774-c732-476f-a289-a698657cd83c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:25.871352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d436774-c732-476f-a289-a698657cd83c admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:25.871978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d436774-c732-476f-a289-a698657cd83c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:25.874293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d436774-c732-476f-a289-a698657cd83c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:25.925475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8b402-fdc4-4f47-a826-a312a52b7bd0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:25.939901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 888/1773] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:25 2022] POST /identity/v3/users => generated 295 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:25.940919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d436774-c732-476f-a289-a698657cd83c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:25.945171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198a043-14f6-4a55-ba70-8a8cdfa08416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:25.953117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198a043-14f6-4a55-ba70-8a8cdfa08416 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:25.953463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198a043-14f6-4a55-ba70-8a8cdfa08416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:25.953737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198a043-14f6-4a55-ba70-8a8cdfa08416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:25.975809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d436774-c732-476f-a289-a698657cd83c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:25.989450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198a043-14f6-4a55-ba70-8a8cdfa08416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:25.995347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198a043-14f6-4a55-ba70-8a8cdfa08416 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:26.017263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10fc0-d6ee-4e8b-9cef-10f3f69e2bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:26.017599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10fc0-d6ee-4e8b-9cef-10f3f69e2bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:26.017879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10fc0-d6ee-4e8b-9cef-10f3f69e2bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:26.018147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10fc0-d6ee-4e8b-9cef-10f3f69e2bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:26.018815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd10fc0-d6ee-4e8b-9cef-10f3f69e2bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:26.020492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd10fc0-d6ee-4e8b-9cef-10f3f69e2bdb admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:26.047120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 886/1774] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:26.091494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 889/1775] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:26.095462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198a043-14f6-4a55-ba70-8a8cdfa08416 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:26.131200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198a043-14f6-4a55-ba70-8a8cdfa08416 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:26.157692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2f9be-29a0-408f-946d-dab641cb17ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:26.176001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f9be-29a0-408f-946d-dab641cb17ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:26.177272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f9be-29a0-408f-946d-dab641cb17ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:26.177560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2f9be-29a0-408f-946d-dab641cb17ee admin admin] PATH_INFO: `/v3/projects/373574a194b744a3b5cd359217b2c616/users/8502badb6d2e4b9f97a5a3f1adf252aa/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:26.243481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2f9be-29a0-408f-946d-dab641cb17ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=373574a194b744a3b5cd359217b2c616, user_id=8502badb6d2e4b9f97a5a3f1adf252aa, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:26.258789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2f9be-29a0-408f-946d-dab641cb17ee admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:26.317394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 887/1776] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:26 2022] PUT /identity/v3/projects/373574a194b744a3b5cd359217b2c616/users/8502badb6d2e4b9f97a5a3f1adf252aa/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:26.318543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10fc0-d6ee-4e8b-9cef-10f3f69e2bdb admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:26.339657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10fc0-d6ee-4e8b-9cef-10f3f69e2bdb admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:26.607931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a192a8-e335-405a-b204-7dfc9c8347ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:26.608784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a192a8-e335-405a-b204-7dfc9c8347ed admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:26.613425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a192a8-e335-405a-b204-7dfc9c8347ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:26.614126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a192a8-e335-405a-b204-7dfc9c8347ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:26.617193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a192a8-e335-405a-b204-7dfc9c8347ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:26.623605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a192a8-e335-405a-b204-7dfc9c8347ed admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:26.661387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 890/1777] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:26.662688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2f9be-29a0-408f-946d-dab641cb17ee admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:26.693932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2f9be-29a0-408f-946d-dab641cb17ee admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:26.722896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:26.723785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:26.728630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:26.729438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] PATH_INFO: `/v3/projects/373574a194b744a3b5cd359217b2c616/users/8502badb6d2e4b9f97a5a3f1adf252aa/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:26.738096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=373574a194b744a3b5cd359217b2c616, user_id=8502badb6d2e4b9f97a5a3f1adf252aa, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:26.749144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:26.794811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 888/1778] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:26 2022] PUT /identity/v3/projects/373574a194b744a3b5cd359217b2c616/users/8502badb6d2e4b9f97a5a3f1adf252aa/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:26.806835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4b08-895f-4525-b835-e1a7d15470f7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:26.807128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4b08-895f-4525-b835-e1a7d15470f7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:26.807406 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4b08-895f-4525-b835-e1a7d15470f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:26.901512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-905e4b08-895f-4525-b835-e1a7d15470f7 None None] MFA Rules not processed for user `8502badb6d2e4b9f97a5a3f1adf252aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:26.904556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-905e4b08-895f-4525-b835-e1a7d15470f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:27.064929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:27.165239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:27.167070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:27.167196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 891/1779] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:27.206239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834477e2-de7e-48ec-96fd-72ff74a3e6aa None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:27.206798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834477e2-de7e-48ec-96fd-72ff74a3e6aa None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:27.207220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834477e2-de7e-48ec-96fd-72ff74a3e6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:27.289144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-834477e2-de7e-48ec-96fd-72ff74a3e6aa None None] MFA Rules not processed for user `e7e5b04101c1403d9b3a311803ac798f`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:27.305615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-834477e2-de7e-48ec-96fd-72ff74a3e6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:27.346825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:27.462117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e78e42-be4a-423c-9276-d77cdec0e179 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:27.480482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:27.481135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:27.485177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:27.485452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:27.530396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 892/1780] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:27.597588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834477e2-de7e-48ec-96fd-72ff74a3e6aa None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:27.599402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-834477e2-de7e-48ec-96fd-72ff74a3e6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:27.656435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:27.657993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:27.691053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 889/1781] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:27.695316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:27.697068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:27.918552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834477e2-de7e-48ec-96fd-72ff74a3e6aa None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:27.926802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4eb355-b1eb-4738-8153-32afea1808a5 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:27.941151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:27.947187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:27.948719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:27.949607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:27.950137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] PATH_INFO: `/v3/projects/25d5d82a8c6c4db79fce9140f0f2aae7` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:27.951996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:27.953843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d5d82a8c6c4db79fce9140f0f2aae7)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:27.956357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:27.959129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] PATH_INFO: `/v3/projects/afb173ec8f4b42349b2c3dbc4de6444a` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:27.963907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb173ec8f4b42349b2c3dbc4de6444a)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:27.964637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:27.968278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:28.197334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d5d82a8c6c4db79fce9140f0f2aae7', 'request_id': 'req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.201057 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb173ec8f4b42349b2c3dbc4de6444a', 'request_id': 'req-87918b65-62ac-483d-bc97-9e8fb2d52b20'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.228707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb173ec8f4b42349b2c3dbc4de6444a', 'request_id': 'req-87918b65-62ac-483d-bc97-9e8fb2d52b20'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.231598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d5d82a8c6c4db79fce9140f0f2aae7', 'request_id': 'req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.258745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb173ec8f4b42349b2c3dbc4de6444a', 'request_id': 'req-87918b65-62ac-483d-bc97-9e8fb2d52b20'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.262041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d5d82a8c6c4db79fce9140f0f2aae7', 'request_id': 'req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.305745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] The token cache is being invalidate because project afb173ec8f4b42349b2c3dbc4de6444a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:28.307539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] The token cache is being invalidate because project 25d5d82a8c6c4db79fce9140f0f2aae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:28.311648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.316807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.317392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.319812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-87918b65-62ac-483d-bc97-9e8fb2d52b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:28.321799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 890/1783] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:27 2022] DELETE /identity/v3/projects/afb173ec8f4b42349b2c3dbc4de6444a => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:28.321941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 893/1783] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:27 2022] DELETE /identity/v3/projects/25d5d82a8c6c4db79fce9140f0f2aae7 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:28.326732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9f153-5b74-4206-83ac-a2075bce0cd3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:28.326732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9f153-5b74-4206-83ac-a2075bce0cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:28.326732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9f153-5b74-4206-83ac-a2075bce0cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:28.328277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:28.332388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:28.371289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:28.381537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c3e9f153-5b74-4206-83ac-a2075bce0cd3 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:28.385321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c3e9f153-5b74-4206-83ac-a2075bce0cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:28.417469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:28.573881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f26a0-cfcd-4752-b904-4b1842ff9d30 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:28.684585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 891/1784] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:28.690613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a90423-2602-4c0c-86bb-9adbc7104084 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:28.691025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a90423-2602-4c0c-86bb-9adbc7104084 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:28.691307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a90423-2602-4c0c-86bb-9adbc7104084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:28.716607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:28.717564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:28.718146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:28.718684 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] PATH_INFO: `/v3/users/a2cebb42d3294de096c84764cbfa272e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:28.720561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2cebb42d3294de096c84764cbfa272e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:28.723836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:28.813810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-56a90423-2602-4c0c-86bb-9adbc7104084 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:28.824789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-56a90423-2602-4c0c-86bb-9adbc7104084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:28.896372 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 892/1785] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:28.897363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a90423-2602-4c0c-86bb-9adbc7104084 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:28.926950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a90423-2602-4c0c-86bb-9adbc7104084 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:28.933930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:28.934602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:28.934880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:28.935148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:28.935808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:28.936176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:28.941055 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:28.969976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 893/1786] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:28:28.971211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:28.999908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb87b42-eecd-46ef-a92e-9580f12c37aa admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:29.000385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2cebb42d3294de096c84764cbfa272e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:29.025666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50502142-bc7c-4b1a-9a1b-b83ed15fd7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:29.026429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502142-bc7c-4b1a-9a1b-b83ed15fd7bd admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:29.026789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502142-bc7c-4b1a-9a1b-b83ed15fd7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:29.027066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502142-bc7c-4b1a-9a1b-b83ed15fd7bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:29.027775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50502142-bc7c-4b1a-9a1b-b83ed15fd7bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:29.043552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50502142-bc7c-4b1a-9a1b-b83ed15fd7bd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:29.080309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2cebb42d3294de096c84764cbfa272e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:29.083842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2cebb42d3294de096c84764cbfa272e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:29.120814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2cebb42d3294de096c84764cbfa272e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:29.133386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 894/1787] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:28 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:29.141907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50502142-bc7c-4b1a-9a1b-b83ed15fd7bd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:29.144328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2cebb42d3294de096c84764cbfa272e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:29.181566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 894/1788] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:28 2022] DELETE /identity/v3/users/a2cebb42d3294de096c84764cbfa272e => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:29.395666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50502142-bc7c-4b1a-9a1b-b83ed15fd7bd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:29.482543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaab4f8-e881-4222-b30f-fae140e15574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:29.484185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaab4f8-e881-4222-b30f-fae140e15574 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:29.485672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaab4f8-e881-4222-b30f-fae140e15574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:29.486228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaab4f8-e881-4222-b30f-fae140e15574 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:29.487250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaab4f8-e881-4222-b30f-fae140e15574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:29.490858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaab4f8-e881-4222-b30f-fae140e15574 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:29.526596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 895/1789] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:29 2022] POST /identity/v3/users => generated 289 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:29.530386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:29.532257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:29.566892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192feed-cc1b-4aa6-8604-34c4979dab9a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:29.580255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbc59c-28ca-4b93-8269-2671d3c40c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:29.581047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc59c-28ca-4b93-8269-2671d3c40c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:29.582052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc59c-28ca-4b93-8269-2671d3c40c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:29.582341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbc59c-28ca-4b93-8269-2671d3c40c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:29.583233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbc59c-28ca-4b93-8269-2671d3c40c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:29.586393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbc59c-28ca-4b93-8269-2671d3c40c00 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:29.619472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 895/1790] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:29.621182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaab4f8-e881-4222-b30f-fae140e15574 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:29.641635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaab4f8-e881-4222-b30f-fae140e15574 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:29.653687 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9b064-929d-437f-9600-433476614e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:29.655264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b064-929d-437f-9600-433476614e12 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:29.655705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b064-929d-437f-9600-433476614e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:29.656169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b064-929d-437f-9600-433476614e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:29.657009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9b064-929d-437f-9600-433476614e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:29.660690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9b064-929d-437f-9600-433476614e12 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:29.673460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:29.692557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 896/1791] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:28:29.698632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbc59c-28ca-4b93-8269-2671d3c40c00 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:29.723916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbc59c-28ca-4b93-8269-2671d3c40c00 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:29.741005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d4d10-f492-4564-a38b-5c8ef5134fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:29.748228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d4d10-f492-4564-a38b-5c8ef5134fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:29.748228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d4d10-f492-4564-a38b-5c8ef5134fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:29.748840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d4d10-f492-4564-a38b-5c8ef5134fb0 admin admin] PATH_INFO: `/v3/projects/7e930cd38d9d4575a5d3abbf40867470/users/312427083d8c4742a61c6ccfd5d6ec5b/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:29.828264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d4d10-f492-4564-a38b-5c8ef5134fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e930cd38d9d4575a5d3abbf40867470, user_id=312427083d8c4742a61c6ccfd5d6ec5b, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:29.847424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d4d10-f492-4564-a38b-5c8ef5134fb0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:29.902553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 896/1792] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:29 2022] PUT /identity/v3/projects/7e930cd38d9d4575a5d3abbf40867470/users/312427083d8c4742a61c6ccfd5d6ec5b/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:29.915019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2a613-de57-4ed6-9580-e6b773f3567f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:29.915313 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2a613-de57-4ed6-9580-e6b773f3567f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:29.916725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2a613-de57-4ed6-9580-e6b773f3567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:30.007678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-66d2a613-de57-4ed6-9580-e6b773f3567f None None] MFA Rules not processed for user `312427083d8c4742a61c6ccfd5d6ec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:30.009106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-66d2a613-de57-4ed6-9580-e6b773f3567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:30.256256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 897/1793] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:30.258764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d4d10-f492-4564-a38b-5c8ef5134fb0 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:30.264950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:30.433294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d4d10-f492-4564-a38b-5c8ef5134fb0 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:30.458930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b17b2-6605-48ce-8a52-1b9a8f3c6c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:30.459964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b17b2-6605-48ce-8a52-1b9a8f3c6c22 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:30.460437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b17b2-6605-48ce-8a52-1b9a8f3c6c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:30.461647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b17b2-6605-48ce-8a52-1b9a8f3c6c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:30.462438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b17b2-6605-48ce-8a52-1b9a8f3c6c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:30.472942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b17b2-6605-48ce-8a52-1b9a8f3c6c22 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:30.590800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 897/1794] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:30 2022] POST /identity/v3/projects => generated 379 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:30.591890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2a613-de57-4ed6-9580-e6b773f3567f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:30.611681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2a613-de57-4ed6-9580-e6b773f3567f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:30.760689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7846-ed69-4e42-b9f4-3d4669d753aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:30.761281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7846-ed69-4e42-b9f4-3d4669d753aa admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:30.761562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7846-ed69-4e42-b9f4-3d4669d753aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:30.761830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd7846-ed69-4e42-b9f4-3d4669d753aa admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:30.762487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd7846-ed69-4e42-b9f4-3d4669d753aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:30.764454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd7846-ed69-4e42-b9f4-3d4669d753aa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:30.809008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 898/1795] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:30 2022] POST /identity/v3/users => generated 290 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:30.809665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b17b2-6605-48ce-8a52-1b9a8f3c6c22 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:30.828984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b17b2-6605-48ce-8a52-1b9a8f3c6c22 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:30.857102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dd360-7a5a-427d-80af-46aac8f651eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:30.857916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dd360-7a5a-427d-80af-46aac8f651eb admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:30.858352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dd360-7a5a-427d-80af-46aac8f651eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:30.858920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dd360-7a5a-427d-80af-46aac8f651eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:30.859442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dd360-7a5a-427d-80af-46aac8f651eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:30.861245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5dd360-7a5a-427d-80af-46aac8f651eb admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:30.895827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 898/1796] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:30.900214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7846-ed69-4e42-b9f4-3d4669d753aa admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:30.914523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd7846-ed69-4e42-b9f4-3d4669d753aa admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:30.938666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362be40e-cccf-4841-a185-e4f8a013a3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:30.939301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362be40e-cccf-4841-a185-e4f8a013a3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:30.939577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362be40e-cccf-4841-a185-e4f8a013a3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:30.939848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362be40e-cccf-4841-a185-e4f8a013a3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:30.940555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362be40e-cccf-4841-a185-e4f8a013a3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:30.942186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362be40e-cccf-4841-a185-e4f8a013a3bc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:30.959965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 899/1797] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:30.961371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dd360-7a5a-427d-80af-46aac8f651eb admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:30.983819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dd360-7a5a-427d-80af-46aac8f651eb admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:31.009577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143e0cc-6cad-471a-a9fb-8666b6155ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:31.013760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e0cc-6cad-471a-a9fb-8666b6155ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:31.014370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e0cc-6cad-471a-a9fb-8666b6155ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:31.020667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e0cc-6cad-471a-a9fb-8666b6155ea9 admin admin] PATH_INFO: `/v3/projects/39ac59435787433c8e3b6dbc91f8512e/users/d9e9b6f685c34b60b4f292cb7f9c6ae9/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:31.085680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143e0cc-6cad-471a-a9fb-8666b6155ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ac59435787433c8e3b6dbc91f8512e, user_id=d9e9b6f685c34b60b4f292cb7f9c6ae9, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:31.089283 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143e0cc-6cad-471a-a9fb-8666b6155ea9 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:31.163388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 899/1798] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:30 2022] PUT /identity/v3/projects/39ac59435787433c8e3b6dbc91f8512e/users/d9e9b6f685c34b60b4f292cb7f9c6ae9/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:31.173687 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e3c1-876e-451c-ae6e-f2ffc962d145 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:31.174070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e3c1-876e-451c-ae6e-f2ffc962d145 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:31.174259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e3c1-876e-451c-ae6e-f2ffc962d145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:31.254164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0f26e3c1-876e-451c-ae6e-f2ffc962d145 None None] MFA Rules not processed for user `d9e9b6f685c34b60b4f292cb7f9c6ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:31.255247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d500a2-e41f-4350-a699-ec48c11ad2c4 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:31.256297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d500a2-e41f-4350-a699-ec48c11ad2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:31.256576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d500a2-e41f-4350-a699-ec48c11ad2c4 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:31.257693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0f26e3c1-876e-451c-ae6e-f2ffc962d145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:31.259011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 900/1799] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:28:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:28:31.270296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748c849-63fa-4c0b-a4bf-90091f8f7bad None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:31.270581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748c849-63fa-4c0b-a4bf-90091f8f7bad None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:31.270857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748c849-63fa-4c0b-a4bf-90091f8f7bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:31.327411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1748c849-63fa-4c0b-a4bf-90091f8f7bad None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:31.331090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1748c849-63fa-4c0b-a4bf-90091f8f7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:31.377297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:31.421331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 900/1800] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:31.425652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26e3c1-876e-451c-ae6e-f2ffc962d145 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:31.478251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 901/1801] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:28:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:31.544916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26e3c1-876e-451c-ae6e-f2ffc962d145 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:31.550905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289e92f-f3d9-4d74-b9b2-c44f5db4b348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:31.551524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e92f-f3d9-4d74-b9b2-c44f5db4b348 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:31.551805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e92f-f3d9-4d74-b9b2-c44f5db4b348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:31.552076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e92f-f3d9-4d74-b9b2-c44f5db4b348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:31.552921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289e92f-f3d9-4d74-b9b2-c44f5db4b348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:31.555800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289e92f-f3d9-4d74-b9b2-c44f5db4b348 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:31.612285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 901/1802] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:31 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:31.614485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748c849-63fa-4c0b-a4bf-90091f8f7bad None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:31.638137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748c849-63fa-4c0b-a4bf-90091f8f7bad None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:31.702186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:31.740514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df664c3-ac77-48ed-a396-d89a53f464e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:31.742014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df664c3-ac77-48ed-a396-d89a53f464e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:31.742898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df664c3-ac77-48ed-a396-d89a53f464e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:31.743469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df664c3-ac77-48ed-a396-d89a53f464e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:31.744402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df664c3-ac77-48ed-a396-d89a53f464e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:31.747344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df664c3-ac77-48ed-a396-d89a53f464e6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:31.803392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 902/1803] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:31 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:31.810244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289e92f-f3d9-4d74-b9b2-c44f5db4b348 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:31.821771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289e92f-f3d9-4d74-b9b2-c44f5db4b348 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:31.828035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:31.828814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:31.829198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:31.829577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:31.830707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:31.832491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:31.846392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 902/1804] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:31.851047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df664c3-ac77-48ed-a396-d89a53f464e6 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:31.867556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df664c3-ac77-48ed-a396-d89a53f464e6 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:31.877186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536924d-fa03-4530-89b4-8fc7561fed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:31.878161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536924d-fa03-4530-89b4-8fc7561fed9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:31.878710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536924d-fa03-4530-89b4-8fc7561fed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:31.879338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536924d-fa03-4530-89b4-8fc7561fed9d admin admin] PATH_INFO: `/v3/projects/04347ee192444ce38f29255226e1e0d0/users/03f402f3977c47f88ecfa4814e53f91c/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:31.916673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536924d-fa03-4530-89b4-8fc7561fed9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=04347ee192444ce38f29255226e1e0d0, user_id=03f402f3977c47f88ecfa4814e53f91c, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:31.921481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536924d-fa03-4530-89b4-8fc7561fed9d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:31.941428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:31.946342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:31.997692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 903/1805] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:31 2022] PUT /identity/v3/projects/04347ee192444ce38f29255226e1e0d0/users/03f402f3977c47f88ecfa4814e53f91c/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:31.998906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536924d-fa03-4530-89b4-8fc7561fed9d admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:32.213286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c1271-3ea2-4334-9d76-513ff3e19e5d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:32.224931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536924d-fa03-4530-89b4-8fc7561fed9d admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:32.248256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752948d-33be-4367-8821-b55388dff893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:32.248892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752948d-33be-4367-8821-b55388dff893 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:32.249173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752948d-33be-4367-8821-b55388dff893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:32.249445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752948d-33be-4367-8821-b55388dff893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:32.250104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752948d-33be-4367-8821-b55388dff893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:32.251727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752948d-33be-4367-8821-b55388dff893 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:32.278907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 904/1806] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:32.280350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752948d-33be-4367-8821-b55388dff893 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:32.300817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752948d-33be-4367-8821-b55388dff893 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:32.317588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419175e9-b5d1-444a-8f5b-ca3294261d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:32.318268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419175e9-b5d1-444a-8f5b-ca3294261d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:32.318796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419175e9-b5d1-444a-8f5b-ca3294261d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:32.318796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419175e9-b5d1-444a-8f5b-ca3294261d60 admin admin] PATH_INFO: `/v3/projects/04347ee192444ce38f29255226e1e0d0/users/03f402f3977c47f88ecfa4814e53f91c/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:32.324977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419175e9-b5d1-444a-8f5b-ca3294261d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04347ee192444ce38f29255226e1e0d0, user_id=03f402f3977c47f88ecfa4814e53f91c, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:32.327708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419175e9-b5d1-444a-8f5b-ca3294261d60 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:32.350144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 905/1807] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:32 2022] PUT /identity/v3/projects/04347ee192444ce38f29255226e1e0d0/users/03f402f3977c47f88ecfa4814e53f91c/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:32.377448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdb9c5-bb91-4f22-a0dd-7d1c020c800d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:32.377713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdb9c5-bb91-4f22-a0dd-7d1c020c800d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:32.377992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdb9c5-bb91-4f22-a0dd-7d1c020c800d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:32.388463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:32.390420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:32.390702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:32.390971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:32.409532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:32.422685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:32.425343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:32.443651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4ecdb9c5-bb91-4f22-a0dd-7d1c020c800d None None] MFA Rules not processed for user `03f402f3977c47f88ecfa4814e53f91c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:32.446137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4ecdb9c5-bb91-4f22-a0dd-7d1c020c800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:32.466556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 903/1808] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:32.468437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:32.470377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:32.516251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a97754-b1b7-4eee-b400-cc0922e57c1c service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:32.660153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 906/1809] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:32.687317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdb9c5-bb91-4f22-a0dd-7d1c020c800d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:32.689478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4ecdb9c5-bb91-4f22-a0dd-7d1c020c800d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:32.733949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:32.734626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:32.734969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:32.735184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] PATH_INFO: `/v3/projects/796008fb76e34a5a9b0b207a91c8d1a2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:32.738325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=796008fb76e34a5a9b0b207a91c8d1a2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:32.744992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:32.748073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:32.858153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdb9c5-bb91-4f22-a0dd-7d1c020c800d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:32.878786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796008fb76e34a5a9b0b207a91c8d1a2', 'request_id': 'req-3a8ccaea-18ad-4e3f-a486-5c35ba711051'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:32.895994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796008fb76e34a5a9b0b207a91c8d1a2', 'request_id': 'req-3a8ccaea-18ad-4e3f-a486-5c35ba711051'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:32.908923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796008fb76e34a5a9b0b207a91c8d1a2', 'request_id': 'req-3a8ccaea-18ad-4e3f-a486-5c35ba711051'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:32.948196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] The token cache is being invalidate because project 796008fb76e34a5a9b0b207a91c8d1a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:32.948522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:32.963280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3a8ccaea-18ad-4e3f-a486-5c35ba711051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:32.966377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 904/1810] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:32 2022] DELETE /identity/v3/projects/796008fb76e34a5a9b0b207a91c8d1a2 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:32.967881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28449088-3cd0-4468-a68a-057a7812643d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:32.968215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28449088-3cd0-4468-a68a-057a7812643d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:32.968672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28449088-3cd0-4468-a68a-057a7812643d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.012070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:33.013859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.014201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.014399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.021870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:33.022053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-28449088-3cd0-4468-a68a-057a7812643d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:33.030218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-28449088-3cd0-4468-a68a-057a7812643d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:33.201079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:33.202664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:33.203014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 905/1811] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:33.212497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e85e1-2686-4de7-8a2a-03b2331542b8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.213006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e85e1-2686-4de7-8a2a-03b2331542b8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.213427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e85e1-2686-4de7-8a2a-03b2331542b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.215326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 907/1812] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:33.240351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-752e85e1-2686-4de7-8a2a-03b2331542b8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:33.241942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-752e85e1-2686-4de7-8a2a-03b2331542b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:33.255780 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 906/1813] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:33.260535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:33.274507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e85e1-2686-4de7-8a2a-03b2331542b8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:33.282524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd26df-6b3f-413e-84b1-9e9646340ce4 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:33.283227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-752e85e1-2686-4de7-8a2a-03b2331542b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:33.295927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:33.296588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.296868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.297156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.297842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:33.298147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:33.299843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:33.301880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e85e1-2686-4de7-8a2a-03b2331542b8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:33.309035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae1f51-1512-43ac-9f9c-07a7a916f675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:33.309899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae1f51-1512-43ac-9f9c-07a7a916f675 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.310286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae1f51-1512-43ac-9f9c-07a7a916f675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.310723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae1f51-1512-43ac-9f9c-07a7a916f675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.312766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3eae1f51-1512-43ac-9f9c-07a7a916f675 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:33.332415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:33.336490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 908/1814] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:28:33.337686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:33.362663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bedeef-efc2-450a-9848-f895cbee3cdd admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:33.376886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af39272-8e46-4975-8075-b543fa8efe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:33.377513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af39272-8e46-4975-8075-b543fa8efe85 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.377795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af39272-8e46-4975-8075-b543fa8efe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.378066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af39272-8e46-4975-8075-b543fa8efe85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.378700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af39272-8e46-4975-8075-b543fa8efe85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:33.380652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af39272-8e46-4975-8075-b543fa8efe85 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:33.407172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae1f51-1512-43ac-9f9c-07a7a916f675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:33.411585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae1f51-1512-43ac-9f9c-07a7a916f675 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:33.432312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 909/1815] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:33 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:33.436488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af39272-8e46-4975-8075-b543fa8efe85 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:33.448442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 907/1816] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:33.511905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f7957-903e-42bb-b627-4cbabeb61e6d None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.512253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f7957-903e-42bb-b627-4cbabeb61e6d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.512536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f7957-903e-42bb-b627-4cbabeb61e6d None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.513910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 908/1817] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:28:33.533442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1cd3-d3f7-4df1-bc10-86e90428774d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.533717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:33.533835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1cd3-d3f7-4df1-bc10-86e90428774d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.534000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1cd3-d3f7-4df1-bc10-86e90428774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.573801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2f0e1cd3-d3f7-4df1-bc10-86e90428774d None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:33.575397 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2f0e1cd3-d3f7-4df1-bc10-86e90428774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:33.585649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af39272-8e46-4975-8075-b543fa8efe85 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:33.606742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449da31c-a86a-40c5-9a7d-2532a8b5e2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:33.607873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449da31c-a86a-40c5-9a7d-2532a8b5e2ba admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.608323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449da31c-a86a-40c5-9a7d-2532a8b5e2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.608653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449da31c-a86a-40c5-9a7d-2532a8b5e2ba admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.611544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449da31c-a86a-40c5-9a7d-2532a8b5e2ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:33.615308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449da31c-a86a-40c5-9a7d-2532a8b5e2ba admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:33.691076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 910/1818] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:33 2022] POST /identity/v3/users => generated 286 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:33.697456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449da31c-a86a-40c5-9a7d-2532a8b5e2ba admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:33.754302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449da31c-a86a-40c5-9a7d-2532a8b5e2ba admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:33.767111 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 909/1819] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:28:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:33.768644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2c30d-37fb-41f4-a241-46a0250200ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:33.769600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2c30d-37fb-41f4-a241-46a0250200ad admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.770152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2c30d-37fb-41f4-a241-46a0250200ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.770559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2c30d-37fb-41f4-a241-46a0250200ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.773140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2c30d-37fb-41f4-a241-46a0250200ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:33.793171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2c30d-37fb-41f4-a241-46a0250200ad admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:33.825269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 911/1820] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:33.829364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e1cd3-d3f7-4df1-bc10-86e90428774d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:33.850628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e1cd3-d3f7-4df1-bc10-86e90428774d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:33.862776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a8db5-a8a8-48fa-b5db-f82337e546a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:33.863495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8db5-a8a8-48fa-b5db-f82337e546a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.863770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8db5-a8a8-48fa-b5db-f82337e546a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.864036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a8db5-a8a8-48fa-b5db-f82337e546a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.864714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a8db5-a8a8-48fa-b5db-f82337e546a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:33.867450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a8db5-a8a8-48fa-b5db-f82337e546a5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:33.886492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 910/1821] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:33.887759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2c30d-37fb-41f4-a241-46a0250200ad admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:33.915613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2c30d-37fb-41f4-a241-46a0250200ad admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:33.923462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:33.940087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:33.943942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:33.946532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] PATH_INFO: `/v3/projects/93993f65a4fa445b88721fd612dc70f1/users/9a143227a4c44060ad9dbdfeba858f7c/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:33.973958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=93993f65a4fa445b88721fd612dc70f1, user_id=9a143227a4c44060ad9dbdfeba858f7c, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:33.979366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:34.019292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 912/1822] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:33 2022] PUT /identity/v3/projects/93993f65a4fa445b88721fd612dc70f1/users/9a143227a4c44060ad9dbdfeba858f7c/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:34.020586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5b38-f1a2-43f9-ad95-c40f79aa3026 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:34.021011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5b38-f1a2-43f9-ad95-c40f79aa3026 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:34.021343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d5b38-f1a2-43f9-ad95-c40f79aa3026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:34.082881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c52d5b38-f1a2-43f9-ad95-c40f79aa3026 None None] MFA Rules not processed for user `9a143227a4c44060ad9dbdfeba858f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:34.084785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c52d5b38-f1a2-43f9-ad95-c40f79aa3026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:34.228674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:34.230389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:34.350392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 911/1823] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:34.365215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5b38-f1a2-43f9-ad95-c40f79aa3026 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:34.511680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92fd8b-403e-421f-aaf3-dffb40fe41eb admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:34.535211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:34.539090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:34.539381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:34.539657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:34.580423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:34.580703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:34.582086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:34.623943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 913/1824] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:34.715009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d5b38-f1a2-43f9-ad95-c40f79aa3026 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:34.732833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633e30c-209e-4e86-9cdd-3d2fa2fe3ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:34.735676 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633e30c-209e-4e86-9cdd-3d2fa2fe3ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:34.736003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633e30c-209e-4e86-9cdd-3d2fa2fe3ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:34.736356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633e30c-209e-4e86-9cdd-3d2fa2fe3ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:34.738262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633e30c-209e-4e86-9cdd-3d2fa2fe3ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:34.746092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633e30c-209e-4e86-9cdd-3d2fa2fe3ae1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:34.852347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:34.854528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:34.888949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 912/1825] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:34 2022] POST /identity/v3/projects => generated 373 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:34.896446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633e30c-209e-4e86-9cdd-3d2fa2fe3ae1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:35.069558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056e1e5-99da-4334-b53b-d163883a5c63 service nova] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:35.151946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633e30c-209e-4e86-9cdd-3d2fa2fe3ae1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:35.174599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5d33d-af71-4147-a7d1-3a92e62e7ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:35.175236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5d33d-af71-4147-a7d1-3a92e62e7ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:35.180122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5d33d-af71-4147-a7d1-3a92e62e7ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:35.180394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5d33d-af71-4147-a7d1-3a92e62e7ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:35.181084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5d33d-af71-4147-a7d1-3a92e62e7ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:35.183301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5d33d-af71-4147-a7d1-3a92e62e7ea3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:35.229220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 913/1826] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:34 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:35.233000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5d33d-af71-4147-a7d1-3a92e62e7ea3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:35.249458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:35.264402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5d33d-af71-4147-a7d1-3a92e62e7ea3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:35.265494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:35.266115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:35.266563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:35.266846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:35.270164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:35.279435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c3396-dc04-4055-acb1-c9330199e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:35.280071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c3396-dc04-4055-acb1-c9330199e9cd admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:35.282202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c3396-dc04-4055-acb1-c9330199e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:35.282504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c3396-dc04-4055-acb1-c9330199e9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:35.283470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c3396-dc04-4055-acb1-c9330199e9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:35.288953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c3396-dc04-4055-acb1-c9330199e9cd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:35.312388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 914/1827] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:28:35.326221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c3396-dc04-4055-acb1-c9330199e9cd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:35.357143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c3396-dc04-4055-acb1-c9330199e9cd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:35.380291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275ce30-9148-4871-a36d-c6a59191a6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:35.383121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275ce30-9148-4871-a36d-c6a59191a6dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:35.383402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275ce30-9148-4871-a36d-c6a59191a6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:35.383678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275ce30-9148-4871-a36d-c6a59191a6dd admin admin] PATH_INFO: `/v3/projects/21a00316dcfa441880d6887c462a55cb/users/939dbff4762f4fc9883bfea1f23f4205/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:35.443512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275ce30-9148-4871-a36d-c6a59191a6dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a00316dcfa441880d6887c462a55cb, user_id=939dbff4762f4fc9883bfea1f23f4205, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:35.446747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275ce30-9148-4871-a36d-c6a59191a6dd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:35.497567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 915/1828] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:35 2022] PUT /identity/v3/projects/21a00316dcfa441880d6887c462a55cb/users/939dbff4762f4fc9883bfea1f23f4205/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:35.499711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275ce30-9148-4871-a36d-c6a59191a6dd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:35.509948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:35.523203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:35.585244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 914/1829] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 733 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:35.658589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275ce30-9148-4871-a36d-c6a59191a6dd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:35.671786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ef981-f9f4-42d2-8758-a18709686383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:35.672600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef981-f9f4-42d2-8758-a18709686383 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:35.672986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef981-f9f4-42d2-8758-a18709686383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:35.673382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ef981-f9f4-42d2-8758-a18709686383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:35.674139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ef981-f9f4-42d2-8758-a18709686383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:35.675944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ef981-f9f4-42d2-8758-a18709686383 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:35.706027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:35.712581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 916/1830] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:35.779068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4c377-e608-4728-af27-638b19122182 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:35.794977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe8df4-a104-4695-aff4-6a06467e84b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:35.796603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8df4-a104-4695-aff4-6a06467e84b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:35.797578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8df4-a104-4695-aff4-6a06467e84b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:35.799009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8df4-a104-4695-aff4-6a06467e84b8 admin admin] PATH_INFO: `/v3/projects/21a00316dcfa441880d6887c462a55cb/users/939dbff4762f4fc9883bfea1f23f4205/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:35.804633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe8df4-a104-4695-aff4-6a06467e84b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a00316dcfa441880d6887c462a55cb, user_id=939dbff4762f4fc9883bfea1f23f4205, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:35.814229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe8df4-a104-4695-aff4-6a06467e84b8 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:35.883494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 915/1831] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:35 2022] PUT /identity/v3/projects/21a00316dcfa441880d6887c462a55cb/users/939dbff4762f4fc9883bfea1f23f4205/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:35.895156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d975aa-6286-4129-826d-c31a3683c39b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:35.895679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d975aa-6286-4129-826d-c31a3683c39b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:35.895788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d975aa-6286-4129-826d-c31a3683c39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:35.905025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:35.997672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-94d975aa-6286-4129-826d-c31a3683c39b None None] MFA Rules not processed for user `939dbff4762f4fc9883bfea1f23f4205`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:35.999992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-94d975aa-6286-4129-826d-c31a3683c39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:36.301153 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 917/1832] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:36.473282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe8df4-a104-4695-aff4-6a06467e84b8 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:36.632693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe8df4-a104-4695-aff4-6a06467e84b8 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:36.642367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61804ea-9e15-4f46-97a7-83e0d0f691f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:36.648214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61804ea-9e15-4f46-97a7-83e0d0f691f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:36.649429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61804ea-9e15-4f46-97a7-83e0d0f691f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:36.650793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61804ea-9e15-4f46-97a7-83e0d0f691f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:36.675250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61804ea-9e15-4f46-97a7-83e0d0f691f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:36.677394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61804ea-9e15-4f46-97a7-83e0d0f691f6 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:36.713572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 916/1833] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:36.984318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d975aa-6286-4129-826d-c31a3683c39b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:36.987029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-94d975aa-6286-4129-826d-c31a3683c39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:37.192957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d975aa-6286-4129-826d-c31a3683c39b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:37.198408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:37.199054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:37.201436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:37.201725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] PATH_INFO: `/v3/users/312427083d8c4742a61c6ccfd5d6ec5b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:37.203325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=312427083d8c4742a61c6ccfd5d6ec5b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:37.205402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:37.282629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312427083d8c4742a61c6ccfd5d6ec5b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.296231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312427083d8c4742a61c6ccfd5d6ec5b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.297791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312427083d8c4742a61c6ccfd5d6ec5b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.316353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312427083d8c4742a61c6ccfd5d6ec5b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.317142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312427083d8c4742a61c6ccfd5d6ec5b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.352032 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d5b3e-4d19-4b4a-a0ee-3eae4d6fb3f9 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:37.352629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 918/1834] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:36 2022] DELETE /identity/v3/users/312427083d8c4742a61c6ccfd5d6ec5b => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:37.352819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d5b3e-4d19-4b4a-a0ee-3eae4d6fb3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:37.352819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d5b3e-4d19-4b4a-a0ee-3eae4d6fb3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:37.376677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-197d5b3e-4d19-4b4a-a0ee-3eae4d6fb3f9 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:37.377971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-197d5b3e-4d19-4b4a-a0ee-3eae4d6fb3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:37.474251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 917/1835] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:37.477039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:37.478523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:37.482673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d5b3e-4d19-4b4a-a0ee-3eae4d6fb3f9 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:37.484313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-197d5b3e-4d19-4b4a-a0ee-3eae4d6fb3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:37.493549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f8ebb-78dc-430f-ace8-709e314ea1a3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:37.499269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:37.500157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:37.500968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:37.501309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] PATH_INFO: `/v3/users/9a143227a4c44060ad9dbdfeba858f7c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:37.503186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a143227a4c44060ad9dbdfeba858f7c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:37.507310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:37.510413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:37.546368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:37.592391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d5b3e-4d19-4b4a-a0ee-3eae4d6fb3f9 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:37.596084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154cb7a-00a0-4019-9d0e-f4ba66b78b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:37.596737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154cb7a-00a0-4019-9d0e-f4ba66b78b1c service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:37.597012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154cb7a-00a0-4019-9d0e-f4ba66b78b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:37.597280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154cb7a-00a0-4019-9d0e-f4ba66b78b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:37.608153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154cb7a-00a0-4019-9d0e-f4ba66b78b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:37.609866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154cb7a-00a0-4019-9d0e-f4ba66b78b1c service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:37.621844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a143227a4c44060ad9dbdfeba858f7c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.625401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 918/1836] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:37.635218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a143227a4c44060ad9dbdfeba858f7c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.636727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a143227a4c44060ad9dbdfeba858f7c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.654253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a143227a4c44060ad9dbdfeba858f7c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.655624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a143227a4c44060ad9dbdfeba858f7c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:37.670363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 919/1837] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:37 2022] DELETE /identity/v3/users/9a143227a4c44060ad9dbdfeba858f7c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:37.675284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6386e-b8b3-45a5-98bc-8d2eb42f422e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:37.678808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6386e-b8b3-45a5-98bc-8d2eb42f422e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:37.679193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6386e-b8b3-45a5-98bc-8d2eb42f422e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:37.707034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-93a6386e-b8b3-45a5-98bc-8d2eb42f422e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:37.708812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-93a6386e-b8b3-45a5-98bc-8d2eb42f422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:37.818727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 919/1838] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:37.841176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:37.843009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:37.925482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da750e-0558-425e-ab8c-c9b1327db003 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:37.933296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31834-cde1-44c5-83be-74f35a2c3400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:37.934073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31834-cde1-44c5-83be-74f35a2c3400 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:37.934426 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31834-cde1-44c5-83be-74f35a2c3400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:37.934756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31834-cde1-44c5-83be-74f35a2c3400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:37.950727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31834-cde1-44c5-83be-74f35a2c3400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:37.954026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31834-cde1-44c5-83be-74f35a2c3400 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:37.967137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 920/1839] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:38.104285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6386e-b8b3-45a5-98bc-8d2eb42f422e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:38.106539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-93a6386e-b8b3-45a5-98bc-8d2eb42f422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:38.127126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6386e-b8b3-45a5-98bc-8d2eb42f422e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:38.150838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:38.151662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:38.152044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:38.152450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] PATH_INFO: `/v3/projects/7e930cd38d9d4575a5d3abbf40867470` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:38.153843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e930cd38d9d4575a5d3abbf40867470)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:38.156276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:38.225675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e930cd38d9d4575a5d3abbf40867470', 'request_id': 'req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.235329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e930cd38d9d4575a5d3abbf40867470', 'request_id': 'req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.244282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e930cd38d9d4575a5d3abbf40867470', 'request_id': 'req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.274111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] The token cache is being invalidate because project 7e930cd38d9d4575a5d3abbf40867470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:38.274838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.275540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.277410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 920/1840] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:38 2022] DELETE /identity/v3/projects/7e930cd38d9d4575a5d3abbf40867470 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:38.281571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31834-cde1-44c5-83be-74f35a2c3400 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:38.291727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31834-cde1-44c5-83be-74f35a2c3400 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:38.397991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:38.398722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:38.399002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:38.399268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] PATH_INFO: `/v3/users/d9e9b6f685c34b60b4f292cb7f9c6ae9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:38.400763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e9b6f685c34b60b4f292cb7f9c6ae9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:38.403252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:38.430839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:38.480926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e9b6f685c34b60b4f292cb7f9c6ae9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.498419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e9b6f685c34b60b4f292cb7f9c6ae9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.499361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e9b6f685c34b60b4f292cb7f9c6ae9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.515442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e9b6f685c34b60b4f292cb7f9c6ae9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.516526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e9b6f685c34b60b4f292cb7f9c6ae9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.536138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 921/1841] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:38 2022] DELETE /identity/v3/users/d9e9b6f685c34b60b4f292cb7f9c6ae9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:38.704478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:38.706168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:38.720668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f211e10-4b8a-4bad-bf3a-6d1ce0ef68e0 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:38.781781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:38.800289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:38.801348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:38.801915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:38.802440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] PATH_INFO: `/v3/projects/93993f65a4fa445b88721fd612dc70f1` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:38.805136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] RBAC: Authorizing `identity:delete_project(project_id=93993f65a4fa445b88721fd612dc70f1)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:38.807981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:38.893270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93993f65a4fa445b88721fd612dc70f1', 'request_id': 'req-ce4997af-e20b-4bbe-b33a-244a0411827a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.901766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93993f65a4fa445b88721fd612dc70f1', 'request_id': 'req-ce4997af-e20b-4bbe-b33a-244a0411827a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.911424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93993f65a4fa445b88721fd612dc70f1', 'request_id': 'req-ce4997af-e20b-4bbe-b33a-244a0411827a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.930708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] The token cache is being invalidate because project 93993f65a4fa445b88721fd612dc70f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:38.930988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.931916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:38.933470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 921/1842] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:38 2022] DELETE /identity/v3/projects/93993f65a4fa445b88721fd612dc70f1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:38.937833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:38.939284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:38.952317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc8c1b-cf6f-49e7-a362-7bcb6d019241 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:39.030341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:39.030983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:39.031262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:39.031794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] PATH_INFO: `/v3/users/939dbff4762f4fc9883bfea1f23f4205` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:39.032684 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=939dbff4762f4fc9883bfea1f23f4205)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:39.034627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:39.093051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939dbff4762f4fc9883bfea1f23f4205'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.105341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939dbff4762f4fc9883bfea1f23f4205'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.106035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939dbff4762f4fc9883bfea1f23f4205'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.110964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:39.112914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:39.126972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939dbff4762f4fc9883bfea1f23f4205'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.127258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939dbff4762f4fc9883bfea1f23f4205'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.139748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4997af-e20b-4bbe-b33a-244a0411827a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:39.143456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 922/1843] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:38 2022] DELETE /identity/v3/users/939dbff4762f4fc9883bfea1f23f4205 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:39.148878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:39.149614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:39.149974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:39.150308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] PATH_INFO: `/v3/projects/39ac59435787433c8e3b6dbc91f8512e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:39.151656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ac59435787433c8e3b6dbc91f8512e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:39.153749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:39.228290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ac59435787433c8e3b6dbc91f8512e', 'request_id': 'req-b7500ae1-9a13-4383-bc89-c6d99a6470ef'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.244361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ac59435787433c8e3b6dbc91f8512e', 'request_id': 'req-b7500ae1-9a13-4383-bc89-c6d99a6470ef'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.250751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ac59435787433c8e3b6dbc91f8512e', 'request_id': 'req-b7500ae1-9a13-4383-bc89-c6d99a6470ef'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.268669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] The token cache is being invalidate because project 39ac59435787433c8e3b6dbc91f8512e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:39.269045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.269804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.271815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 922/1844] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:39 2022] DELETE /identity/v3/projects/39ac59435787433c8e3b6dbc91f8512e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:39.276418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:39.278231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:39.319732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:39.414592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393127dc-99e4-4e64-8104-410b8b40ac30 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:39.484038 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:39.484712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:39.484993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:39.485263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] PATH_INFO: `/v3/users/03f402f3977c47f88ecfa4814e53f91c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:39.486414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f402f3977c47f88ecfa4814e53f91c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:39.488590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:39.574761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:39.575789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:39.578997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f402f3977c47f88ecfa4814e53f91c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.591836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7500ae1-9a13-4383-bc89-c6d99a6470ef admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:39.609672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f402f3977c47f88ecfa4814e53f91c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.610690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f402f3977c47f88ecfa4814e53f91c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.619465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:39.620685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:39.621162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:39.621638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] PATH_INFO: `/v3/projects/21a00316dcfa441880d6887c462a55cb` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:39.623567 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a00316dcfa441880d6887c462a55cb)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:39.625731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:39.657089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f402f3977c47f88ecfa4814e53f91c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.657855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-572f17c1-f153-4705-95bd-b2600a1b31c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f402f3977c47f88ecfa4814e53f91c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.673926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 923/1845] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:39 2022] DELETE /identity/v3/users/03f402f3977c47f88ecfa4814e53f91c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:39.702399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a00316dcfa441880d6887c462a55cb', 'request_id': 'req-62f21f98-5556-443c-8a80-765267579c33'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.710545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a00316dcfa441880d6887c462a55cb', 'request_id': 'req-62f21f98-5556-443c-8a80-765267579c33'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.716404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:39.717593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a00316dcfa441880d6887c462a55cb', 'request_id': 'req-62f21f98-5556-443c-8a80-765267579c33'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.741860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] The token cache is being invalidate because project 21a00316dcfa441880d6887c462a55cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:39.742442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.744093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-62f21f98-5556-443c-8a80-765267579c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:39.760243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 923/1846] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:39 2022] DELETE /identity/v3/projects/21a00316dcfa441880d6887c462a55cb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:39.762972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edb812-bb99-4fc5-b1b4-faa28b5dd9c2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:39.763298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edb812-bb99-4fc5-b1b4-faa28b5dd9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:39.763727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edb812-bb99-4fc5-b1b4-faa28b5dd9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:39.787411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-46edb812-bb99-4fc5-b1b4-faa28b5dd9c2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:39.790149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46edb812-bb99-4fc5-b1b4-faa28b5dd9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:39.859489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:39.900268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 924/1847] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:39.916789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4c7b8-dfc6-4780-ba15-dc28a9bd6bb5 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:39.918155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4c7b8-dfc6-4780-ba15-dc28a9bd6bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:39.918585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4c7b8-dfc6-4780-ba15-dc28a9bd6bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:39.943697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-10f4c7b8-dfc6-4780-ba15-dc28a9bd6bb5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:39.945804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-10f4c7b8-dfc6-4780-ba15-dc28a9bd6bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:39.960213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 924/1848] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:39.963251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edb812-bb99-4fc5-b1b4-faa28b5dd9c2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:39.979859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edb812-bb99-4fc5-b1b4-faa28b5dd9c2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:39.984865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:39.986378 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:39.986807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:39.987226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:39.988329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:39.989707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:39.992541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.008343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 925/1849] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:28:40.013862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4c7b8-dfc6-4780-ba15-dc28a9bd6bb5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.026862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4c7b8-dfc6-4780-ba15-dc28a9bd6bb5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.036515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916eb30-dc89-4e2a-9aa8-edd7d8444233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.036805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.037447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916eb30-dc89-4e2a-9aa8-edd7d8444233 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.037957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916eb30-dc89-4e2a-9aa8-edd7d8444233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.038920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:40.044076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916eb30-dc89-4e2a-9aa8-edd7d8444233 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.044076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916eb30-dc89-4e2a-9aa8-edd7d8444233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.044076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916eb30-dc89-4e2a-9aa8-edd7d8444233 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.062854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cc4a0-6c1f-4e29-877f-c30889365ec5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.096576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.097230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.097513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.098132 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] PATH_INFO: `/v3/projects/04347ee192444ce38f29255226e1e0d0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.099669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04347ee192444ce38f29255226e1e0d0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.103801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.109116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 925/1850] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:40.112719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916eb30-dc89-4e2a-9aa8-edd7d8444233 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.177549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04347ee192444ce38f29255226e1e0d0', 'request_id': 'req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:40.192256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04347ee192444ce38f29255226e1e0d0', 'request_id': 'req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:40.205015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04347ee192444ce38f29255226e1e0d0', 'request_id': 'req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:40.231796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] The token cache is being invalidate because project 04347ee192444ce38f29255226e1e0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:40.232444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:40.233425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0a3145ef-d9d6-4e66-bfdc-ef133e7741a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:40.235355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 926/1851] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:40 2022] DELETE /identity/v3/projects/04347ee192444ce38f29255226e1e0d0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:40.242814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14affe8c-857e-4fb9-8f0c-b4a897139178 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.243594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14affe8c-857e-4fb9-8f0c-b4a897139178 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.243951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14affe8c-857e-4fb9-8f0c-b4a897139178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.247245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916eb30-dc89-4e2a-9aa8-edd7d8444233 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.285886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-14affe8c-857e-4fb9-8f0c-b4a897139178 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:40.290319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-14affe8c-857e-4fb9-8f0c-b4a897139178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:40.375992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.376724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.377076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.377416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.378775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.385922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 927/1852] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:40.386438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.389248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c9b33-b1dd-454c-9ed1-6320e49f9b43 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.389533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c9b33-b1dd-454c-9ed1-6320e49f9b43 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.389814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c9b33-b1dd-454c-9ed1-6320e49f9b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.413252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 926/1853] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/users => generated 283 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:40.418703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.420818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:40.422011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3e9c9b33-b1dd-454c-9ed1-6320e49f9b43 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:40.423195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3e9c9b33-b1dd-454c-9ed1-6320e49f9b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:40.438495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 928/1854] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:40.439247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786de730-fa85-4526-976d-6dcdbddb5c16 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.439581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c9b33-b1dd-454c-9ed1-6320e49f9b43 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.445967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c7792-8361-4665-9dd0-f6a58748bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.447444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c7792-8361-4665-9dd0-f6a58748bb10 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.447818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c7792-8361-4665-9dd0-f6a58748bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.448086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c7792-8361-4665-9dd0-f6a58748bb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.448777 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c7792-8361-4665-9dd0-f6a58748bb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.450627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c9b33-b1dd-454c-9ed1-6320e49f9b43 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.450729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c7792-8361-4665-9dd0-f6a58748bb10 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.457782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.458494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.460020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.460020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.461281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:40.461674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.463328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 927/1855] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:40.466834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.467096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c7792-8361-4665-9dd0-f6a58748bb10 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.479459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c7792-8361-4665-9dd0-f6a58748bb10 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.485842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 929/1856] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:28:40.486976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.498059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd8a68-d8c8-45b3-9fbd-74ba3f06ad61 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.499612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91589c-7c3d-4c3a-bcd4-aa621208d575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.500393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91589c-7c3d-4c3a-bcd4-aa621208d575 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.500549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91589c-7c3d-4c3a-bcd4-aa621208d575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.500799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91589c-7c3d-4c3a-bcd4-aa621208d575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.501473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91589c-7c3d-4c3a-bcd4-aa621208d575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.503668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a18ce-a0e2-4d4a-ad03-b6164eaaff6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.504413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a18ce-a0e2-4d4a-ad03-b6164eaaff6c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.504816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a18ce-a0e2-4d4a-ad03-b6164eaaff6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.505347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a18ce-a0e2-4d4a-ad03-b6164eaaff6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.506071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a18ce-a0e2-4d4a-ad03-b6164eaaff6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.508335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91589c-7c3d-4c3a-bcd4-aa621208d575 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.509291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a18ce-a0e2-4d4a-ad03-b6164eaaff6c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.529563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 928/1857] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:40.533473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91589c-7c3d-4c3a-bcd4-aa621208d575 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.550968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91589c-7c3d-4c3a-bcd4-aa621208d575 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.556797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 930/1858] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:40.560505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a18ce-a0e2-4d4a-ad03-b6164eaaff6c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.578194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:40.590783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:40.618290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d281-400e-432e-a2f8-467bcb284c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.618951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d281-400e-432e-a2f8-467bcb284c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.619231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d281-400e-432e-a2f8-467bcb284c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.619502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848d281-400e-432e-a2f8-467bcb284c8c admin admin] PATH_INFO: `/v3/projects/8cd46db5c1854bf0a170bd4318d8a38c/users/a330531549e844998161fefe8d2a614a/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.627059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a18ce-a0e2-4d4a-ad03-b6164eaaff6c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.631814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f83562-50ca-448f-8b6a-2a6bd552af3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.632461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f83562-50ca-448f-8b6a-2a6bd552af3f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.632738 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f83562-50ca-448f-8b6a-2a6bd552af3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.633010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f83562-50ca-448f-8b6a-2a6bd552af3f admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.633631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f83562-50ca-448f-8b6a-2a6bd552af3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.635609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f83562-50ca-448f-8b6a-2a6bd552af3f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.636529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848d281-400e-432e-a2f8-467bcb284c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd46db5c1854bf0a170bd4318d8a38c, user_id=a330531549e844998161fefe8d2a614a, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.639155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848d281-400e-432e-a2f8-467bcb284c8c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.654606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 929/1859] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:40 2022] PUT /identity/v3/projects/8cd46db5c1854bf0a170bd4318d8a38c/users/a330531549e844998161fefe8d2a614a/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:40.659617 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 931/1860] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:40.664370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d281-400e-432e-a2f8-467bcb284c8c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.666522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89717c-e6c1-4fb7-a54c-c8142d1ae64c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.666916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89717c-e6c1-4fb7-a54c-c8142d1ae64c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.667190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89717c-e6c1-4fb7-a54c-c8142d1ae64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.700900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4e89717c-e6c1-4fb7-a54c-c8142d1ae64c None None] MFA Rules not processed for user `a330531549e844998161fefe8d2a614a`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:40.702116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4e89717c-e6c1-4fb7-a54c-c8142d1ae64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:40.744143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848d281-400e-432e-a2f8-467bcb284c8c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.749076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4d4f4-5ed1-44ab-a07b-8dee0166e78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.749629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4d4f4-5ed1-44ab-a07b-8dee0166e78e admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.749903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4d4f4-5ed1-44ab-a07b-8dee0166e78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.750176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4d4f4-5ed1-44ab-a07b-8dee0166e78e admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.750847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4d4f4-5ed1-44ab-a07b-8dee0166e78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.752466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4d4f4-5ed1-44ab-a07b-8dee0166e78e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.767002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 930/1861] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:40.768745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4d4f4-5ed1-44ab-a07b-8dee0166e78e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.781706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4d4f4-5ed1-44ab-a07b-8dee0166e78e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.781914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 932/1862] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:40.786309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d7230-5ba9-4cc3-91e8-b15f1a985bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.786459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89717c-e6c1-4fb7-a54c-c8142d1ae64c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.787124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7230-5ba9-4cc3-91e8-b15f1a985bab admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.787345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7230-5ba9-4cc3-91e8-b15f1a985bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.787694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d7230-5ba9-4cc3-91e8-b15f1a985bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.788383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d7230-5ba9-4cc3-91e8-b15f1a985bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.790203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d7230-5ba9-4cc3-91e8-b15f1a985bab admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.797972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89717c-e6c1-4fb7-a54c-c8142d1ae64c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.804068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f2cd2-900f-4b8b-a065-7258aba84d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.804720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2cd2-900f-4b8b-a065-7258aba84d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.804999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2cd2-900f-4b8b-a065-7258aba84d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.805272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f2cd2-900f-4b8b-a065-7258aba84d8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.805912 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f2cd2-900f-4b8b-a065-7258aba84d8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.808080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f2cd2-900f-4b8b-a065-7258aba84d8a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.808920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 931/1863] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:28:40.813085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d7230-5ba9-4cc3-91e8-b15f1a985bab admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.824298 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d7230-5ba9-4cc3-91e8-b15f1a985bab admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.829193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bd547-2949-4013-a6d6-a01942671483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.829921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd547-2949-4013-a6d6-a01942671483 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.830153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd547-2949-4013-a6d6-a01942671483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.830542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bd547-2949-4013-a6d6-a01942671483 admin admin] PATH_INFO: `/v3/projects/614cc401a97144139a50f76b9b243bc9/users/267cd1bf56534eef92049c43d68bfc5e/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.847032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 933/1864] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:40.848126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f2cd2-900f-4b8b-a065-7258aba84d8a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:40.854115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bd547-2949-4013-a6d6-a01942671483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614cc401a97144139a50f76b9b243bc9, user_id=267cd1bf56534eef92049c43d68bfc5e, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.857769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bd547-2949-4013-a6d6-a01942671483 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.874902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 932/1865] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:40 2022] PUT /identity/v3/projects/614cc401a97144139a50f76b9b243bc9/users/267cd1bf56534eef92049c43d68bfc5e/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:40.887922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ad26-c3bb-46ab-a7d5-49196c115f49 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.888361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ad26-c3bb-46ab-a7d5-49196c115f49 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.888708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28ad26-c3bb-46ab-a7d5-49196c115f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.917824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f2cd2-900f-4b8b-a065-7258aba84d8a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:40.930576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-fb28ad26-c3bb-46ab-a7d5-49196c115f49 None None] MFA Rules not processed for user `267cd1bf56534eef92049c43d68bfc5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:40.932125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fb28ad26-c3bb-46ab-a7d5-49196c115f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:40.937668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:40.978858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b464a78-e36d-4191-bd86-eb83d958891c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:40.979490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b464a78-e36d-4191-bd86-eb83d958891c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:40.979765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b464a78-e36d-4191-bd86-eb83d958891c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:40.980032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b464a78-e36d-4191-bd86-eb83d958891c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:40.980678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b464a78-e36d-4191-bd86-eb83d958891c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:40.982566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b464a78-e36d-4191-bd86-eb83d958891c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:40.997793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 933/1866] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:41.002011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28ad26-c3bb-46ab-a7d5-49196c115f49 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.003833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 934/1867] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:40 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:41.007947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b464a78-e36d-4191-bd86-eb83d958891c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.013003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28ad26-c3bb-46ab-a7d5-49196c115f49 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.017201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b464a78-e36d-4191-bd86-eb83d958891c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.017799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf8986-984c-4efa-9796-9cebf4fc2a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.018391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8986-984c-4efa-9796-9cebf4fc2a99 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.018670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8986-984c-4efa-9796-9cebf4fc2a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.018940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf8986-984c-4efa-9796-9cebf4fc2a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.019561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf8986-984c-4efa-9796-9cebf4fc2a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.021487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf8986-984c-4efa-9796-9cebf4fc2a99 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.022533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09c016-070e-4d8f-9581-8a36559760c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.023150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09c016-070e-4d8f-9581-8a36559760c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.023423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09c016-070e-4d8f-9581-8a36559760c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.023689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09c016-070e-4d8f-9581-8a36559760c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.024377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09c016-070e-4d8f-9581-8a36559760c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.025929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa09c016-070e-4d8f-9581-8a36559760c9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.027647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:41.037882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 935/1868] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:41.042485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09c016-070e-4d8f-9581-8a36559760c9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.053732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa09c016-070e-4d8f-9581-8a36559760c9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.059622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75095ccb-a117-4b1a-bbac-50e4c619b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.060448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75095ccb-a117-4b1a-bbac-50e4c619b2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.060740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75095ccb-a117-4b1a-bbac-50e4c619b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.060949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75095ccb-a117-4b1a-bbac-50e4c619b2f1 admin admin] PATH_INFO: `/v3/projects/be4256496931421e909d422d298b3d19/users/4ad6d49d59bd43b7a537685db503e82d/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.071091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 934/1869] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:41 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:41.074786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf8986-984c-4efa-9796-9cebf4fc2a99 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.079700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75095ccb-a117-4b1a-bbac-50e4c619b2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be4256496931421e909d422d298b3d19, user_id=4ad6d49d59bd43b7a537685db503e82d, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.082753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75095ccb-a117-4b1a-bbac-50e4c619b2f1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.102081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 936/1870] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:41 2022] PUT /identity/v3/projects/be4256496931421e909d422d298b3d19/users/4ad6d49d59bd43b7a537685db503e82d/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:41.103107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75095ccb-a117-4b1a-bbac-50e4c619b2f1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.145004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf8986-984c-4efa-9796-9cebf4fc2a99 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.177877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75095ccb-a117-4b1a-bbac-50e4c619b2f1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.183849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26c9dc-e5ae-4660-8dca-a32b03ccd0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.184578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26c9dc-e5ae-4660-8dca-a32b03ccd0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.184958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26c9dc-e5ae-4660-8dca-a32b03ccd0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.185328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26c9dc-e5ae-4660-8dca-a32b03ccd0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.186463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26c9dc-e5ae-4660-8dca-a32b03ccd0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.187824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26c9dc-e5ae-4660-8dca-a32b03ccd0a7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.207569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 937/1871] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:28:41.208449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26c9dc-e5ae-4660-8dca-a32b03ccd0a7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.220949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26c9dc-e5ae-4660-8dca-a32b03ccd0a7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.228266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cc89e-802d-4e91-a1bf-b5c46fa4c982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.229696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc89e-802d-4e91-a1bf-b5c46fa4c982 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.229973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc89e-802d-4e91-a1bf-b5c46fa4c982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.230237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc89e-802d-4e91-a1bf-b5c46fa4c982 admin admin] PATH_INFO: `/v3/projects/be4256496931421e909d422d298b3d19/users/4ad6d49d59bd43b7a537685db503e82d/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.232696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cc89e-802d-4e91-a1bf-b5c46fa4c982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be4256496931421e909d422d298b3d19, user_id=4ad6d49d59bd43b7a537685db503e82d, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.234656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f588ae4-4009-4d84-bb6e-2a076aaa8677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.235519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f588ae4-4009-4d84-bb6e-2a076aaa8677 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.235628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582cc89e-802d-4e91-a1bf-b5c46fa4c982 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.236090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f588ae4-4009-4d84-bb6e-2a076aaa8677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.236747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f588ae4-4009-4d84-bb6e-2a076aaa8677 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.237579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f588ae4-4009-4d84-bb6e-2a076aaa8677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.240902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f588ae4-4009-4d84-bb6e-2a076aaa8677 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.248521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 938/1872] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:41 2022] PUT /identity/v3/projects/be4256496931421e909d422d298b3d19/users/4ad6d49d59bd43b7a537685db503e82d/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:41.261514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699eca-84a2-491f-b5aa-284a5b527428 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.261800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699eca-84a2-491f-b5aa-284a5b527428 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.262261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46699eca-84a2-491f-b5aa-284a5b527428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.275251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 935/1873] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:41 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:41.279495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f588ae4-4009-4d84-bb6e-2a076aaa8677 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.290826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f588ae4-4009-4d84-bb6e-2a076aaa8677 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.296913 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-46699eca-84a2-491f-b5aa-284a5b527428 None None] MFA Rules not processed for user `4ad6d49d59bd43b7a537685db503e82d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:41.298134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46699eca-84a2-491f-b5aa-284a5b527428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:41.351500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:41.382215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 939/1874] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:41.391919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891b3da-3a1d-4ec5-967b-ff30414ac4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.392639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891b3da-3a1d-4ec5-967b-ff30414ac4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.393108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891b3da-3a1d-4ec5-967b-ff30414ac4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.393444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891b3da-3a1d-4ec5-967b-ff30414ac4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.394473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891b3da-3a1d-4ec5-967b-ff30414ac4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.395972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46699eca-84a2-491f-b5aa-284a5b527428 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.396620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891b3da-3a1d-4ec5-967b-ff30414ac4e4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.397950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46699eca-84a2-491f-b5aa-284a5b527428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:41.412314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 936/1875] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:41.416674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891b3da-3a1d-4ec5-967b-ff30414ac4e4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.427299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891b3da-3a1d-4ec5-967b-ff30414ac4e4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.433989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a6ce4-8c31-49b9-a88e-310ad529f00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.434903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a6ce4-8c31-49b9-a88e-310ad529f00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.435280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a6ce4-8c31-49b9-a88e-310ad529f00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.435675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a6ce4-8c31-49b9-a88e-310ad529f00b admin admin] PATH_INFO: `/v3/projects/3c4ef198141342ba9e19d46cbf10f9ad/users/b8448e0de42d49e69ca72628d85c3934/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.461235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a6ce4-8c31-49b9-a88e-310ad529f00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4ef198141342ba9e19d46cbf10f9ad, user_id=b8448e0de42d49e69ca72628d85c3934, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.465666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a6ce4-8c31-49b9-a88e-310ad529f00b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.472050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:41.490451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46699eca-84a2-491f-b5aa-284a5b527428 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.494405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 937/1876] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:41 2022] PUT /identity/v3/projects/3c4ef198141342ba9e19d46cbf10f9ad/users/b8448e0de42d49e69ca72628d85c3934/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:41.498147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a6ce4-8c31-49b9-a88e-310ad529f00b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.580140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aad70-a747-4f58-945e-5030394a1ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.580937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aad70-a747-4f58-945e-5030394a1ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.581851 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aad70-a747-4f58-945e-5030394a1ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.582356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aad70-a747-4f58-945e-5030394a1ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.606465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a6ce4-8c31-49b9-a88e-310ad529f00b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.617363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497dcbd-333a-4da9-85c2-5dd06cdabe5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.617991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dcbd-333a-4da9-85c2-5dd06cdabe5c admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.618273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dcbd-333a-4da9-85c2-5dd06cdabe5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.618542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dcbd-333a-4da9-85c2-5dd06cdabe5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.619343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497dcbd-333a-4da9-85c2-5dd06cdabe5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.621025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497dcbd-333a-4da9-85c2-5dd06cdabe5c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.633738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 938/1877] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:41.637704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497dcbd-333a-4da9-85c2-5dd06cdabe5c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.650112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497dcbd-333a-4da9-85c2-5dd06cdabe5c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.655143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da60b9-e34e-4d69-b528-465d938b021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.656048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da60b9-e34e-4d69-b528-465d938b021a admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.656396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da60b9-e34e-4d69-b528-465d938b021a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.656676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da60b9-e34e-4d69-b528-465d938b021a admin admin] PATH_INFO: `/v3/projects/3c4ef198141342ba9e19d46cbf10f9ad/users/b8448e0de42d49e69ca72628d85c3934/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.658624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da60b9-e34e-4d69-b528-465d938b021a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c4ef198141342ba9e19d46cbf10f9ad, user_id=b8448e0de42d49e69ca72628d85c3934, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.661588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da60b9-e34e-4d69-b528-465d938b021a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.674517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 939/1878] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:41 2022] PUT /identity/v3/projects/3c4ef198141342ba9e19d46cbf10f9ad/users/b8448e0de42d49e69ca72628d85c3934/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:41.680238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4aad70-a747-4f58-945e-5030394a1ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:41.682052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4aad70-a747-4f58-945e-5030394a1ca5 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:41.685545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f5c3-10f6-45ad-85f0-bc308059d3e3 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.685836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f5c3-10f6-45ad-85f0-bc308059d3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.686351 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417f5c3-10f6-45ad-85f0-bc308059d3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.697066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 940/1879] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:41.724600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1417f5c3-10f6-45ad-85f0-bc308059d3e3 None None] MFA Rules not processed for user `b8448e0de42d49e69ca72628d85c3934`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:41.726840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1417f5c3-10f6-45ad-85f0-bc308059d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:41.821592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aad70-a747-4f58-945e-5030394a1ca5 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.827269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 940/1880] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:41.835521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aad70-a747-4f58-945e-5030394a1ca5 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.839383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417f5c3-10f6-45ad-85f0-bc308059d3e3 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:41.841162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1417f5c3-10f6-45ad-85f0-bc308059d3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:41.910365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:41.919235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a2528-2ee5-4387-a66c-e2305a62b166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.919870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a2528-2ee5-4387-a66c-e2305a62b166 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.921494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417f5c3-10f6-45ad-85f0-bc308059d3e3 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:41.922477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a2528-2ee5-4387-a66c-e2305a62b166 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.922755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a2528-2ee5-4387-a66c-e2305a62b166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.936427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518507-a9a8-43ab-bfd1-f8ac304e8fc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:41.936988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518507-a9a8-43ab-bfd1-f8ac304e8fc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:41.937264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518507-a9a8-43ab-bfd1-f8ac304e8fc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:41.937532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518507-a9a8-43ab-bfd1-f8ac304e8fc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:41.968085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:42.015156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16518507-a9a8-43ab-bfd1-f8ac304e8fc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:42.016680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16518507-a9a8-43ab-bfd1-f8ac304e8fc0 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:42.025624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a2528-2ee5-4387-a66c-e2305a62b166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:42.027300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a2528-2ee5-4387-a66c-e2305a62b166 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:42.028938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 941/1881] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:42.040680 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 941/1882] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:45.010643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518507-a9a8-43ab-bfd1-f8ac304e8fc0 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:45.078974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518507-a9a8-43ab-bfd1-f8ac304e8fc0 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:45.083504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:45.084161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:45.084912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:45.084912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] PATH_INFO: `/v3/users/a330531549e844998161fefe8d2a614a` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:45.085911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a330531549e844998161fefe8d2a614a)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:45.087829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:45.132456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a330531549e844998161fefe8d2a614a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:45.146967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a330531549e844998161fefe8d2a614a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:45.147705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a330531549e844998161fefe8d2a614a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:45.155475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a330531549e844998161fefe8d2a614a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:45.165440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a330531549e844998161fefe8d2a614a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:45.172820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 942/1883] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:45 2022] DELETE /identity/v3/users/a330531549e844998161fefe8d2a614a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:45.177310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d856-011c-4f20-8b53-f431f5ed02a5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:45.177803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d856-011c-4f20-8b53-f431f5ed02a5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:45.178546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d856-011c-4f20-8b53-f431f5ed02a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:45.198629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c580d856-011c-4f20-8b53-f431f5ed02a5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:45.199904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c580d856-011c-4f20-8b53-f431f5ed02a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:45.268614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 942/1884] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:45.279762 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:45.281377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:45.349606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:45.368612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b2cf-408d-48b9-b245-0797488a7a49 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:45.372539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fef61d-5a04-4610-812d-d7df72b7fa58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:45.373408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fef61d-5a04-4610-812d-d7df72b7fa58 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:45.373782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fef61d-5a04-4610-812d-d7df72b7fa58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:45.374508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fef61d-5a04-4610-812d-d7df72b7fa58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:45.389129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fef61d-5a04-4610-812d-d7df72b7fa58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:45.390680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fef61d-5a04-4610-812d-d7df72b7fa58 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:45.403701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 943/1885] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:28:46.885173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580d856-011c-4f20-8b53-f431f5ed02a5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:46.888317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c580d856-011c-4f20-8b53-f431f5ed02a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:46.932512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580d856-011c-4f20-8b53-f431f5ed02a5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:46.943182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:46.944785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:46.946046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:46.946849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] PATH_INFO: `/v3/projects/8cd46db5c1854bf0a170bd4318d8a38c` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:46.949539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd46db5c1854bf0a170bd4318d8a38c)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:46.952803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:47.008679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e4680-fe90-4fd5-b7b9-e358c99e1d2f None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:47.009461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e4680-fe90-4fd5-b7b9-e358c99e1d2f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:47.009881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e4680-fe90-4fd5-b7b9-e358c99e1d2f None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:47.011537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 944/1886] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 11 16:28:47.072622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ad0b4-f42f-4f01-82fc-f0555da7bf54 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:47.073220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ad0b4-f42f-4f01-82fc-f0555da7bf54 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:47.073639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ad0b4-f42f-4f01-82fc-f0555da7bf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:47.127255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c58ad0b4-f42f-4f01-82fc-f0555da7bf54 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:47.131994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd46db5c1854bf0a170bd4318d8a38c', 'request_id': 'req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.132382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c58ad0b4-f42f-4f01-82fc-f0555da7bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:47.149241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd46db5c1854bf0a170bd4318d8a38c', 'request_id': 'req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.163623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd46db5c1854bf0a170bd4318d8a38c', 'request_id': 'req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.213179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] The token cache is being invalidate because project 8cd46db5c1854bf0a170bd4318d8a38c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:47.213495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.216979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.218718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 943/1887] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:46 2022] DELETE /identity/v3/projects/8cd46db5c1854bf0a170bd4318d8a38c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:47.222699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:47.224222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:47.284203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:47.364586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 945/1888] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:28:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:47.394910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4ec6-285c-4c7a-9bcf-bf9760b4418e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:47.411760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:47.414764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:47.417671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:47.418124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] PATH_INFO: `/v3/users/4ad6d49d59bd43b7a537685db503e82d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:47.422040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad6d49d59bd43b7a537685db503e82d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:47.429469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:47.554548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad6d49d59bd43b7a537685db503e82d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.575050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad6d49d59bd43b7a537685db503e82d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.575727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad6d49d59bd43b7a537685db503e82d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.601166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad6d49d59bd43b7a537685db503e82d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.601944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad6d49d59bd43b7a537685db503e82d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:47.641826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 944/1889] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:47 2022] DELETE /identity/v3/users/4ad6d49d59bd43b7a537685db503e82d => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:47.947822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ad0b4-f42f-4f01-82fc-f0555da7bf54 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:47.949678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c58ad0b4-f42f-4f01-82fc-f0555da7bf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:47.966574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ad0b4-f42f-4f01-82fc-f0555da7bf54 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:48.047868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453da0e6-ce92-4e61-bc5c-4cd2938eaa28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:48.048578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453da0e6-ce92-4e61-bc5c-4cd2938eaa28 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.048786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453da0e6-ce92-4e61-bc5c-4cd2938eaa28 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.049229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453da0e6-ce92-4e61-bc5c-4cd2938eaa28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.055621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-453da0e6-ce92-4e61-bc5c-4cd2938eaa28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:48.068064 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453da0e6-ce92-4e61-bc5c-4cd2938eaa28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:48.069744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453da0e6-ce92-4e61-bc5c-4cd2938eaa28 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:48.086526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 946/1890] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:48.110876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:48.112377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:48.237527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c587603-054d-4900-9f8f-035408d317bb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:48.245672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:48.246326 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.246600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.246871 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] PATH_INFO: `/v3/projects/be4256496931421e909d422d298b3d19` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.248003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=be4256496931421e909d422d298b3d19)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:48.249952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:48.305598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:48.307335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be4256496931421e909d422d298b3d19', 'request_id': 'req-09713d50-a2d1-4987-8977-b5c6e1dc7dda'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:48.320359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be4256496931421e909d422d298b3d19', 'request_id': 'req-09713d50-a2d1-4987-8977-b5c6e1dc7dda'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:48.329307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be4256496931421e909d422d298b3d19', 'request_id': 'req-09713d50-a2d1-4987-8977-b5c6e1dc7dda'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:48.351220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] The token cache is being invalidate because project be4256496931421e909d422d298b3d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:48.351576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:48.352771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-09713d50-a2d1-4987-8977-b5c6e1dc7dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:48.354837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 945/1891] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:48 2022] DELETE /identity/v3/projects/be4256496931421e909d422d298b3d19 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:48.359775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb191-7a10-4090-a7c3-e94d97ce971f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.360030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb191-7a10-4090-a7c3-e94d97ce971f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.360547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efeb191-7a10-4090-a7c3-e94d97ce971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.387667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-8efeb191-7a10-4090-a7c3-e94d97ce971f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:48.390450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8efeb191-7a10-4090-a7c3-e94d97ce971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:48.465964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:48.476185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 947/1892] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:48.483720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7157cd6-7ddd-4c57-9953-31e112d28409 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.484027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7157cd6-7ddd-4c57-9953-31e112d28409 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.484371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7157cd6-7ddd-4c57-9953-31e112d28409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.504707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a7157cd6-7ddd-4c57-9953-31e112d28409 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:48.506407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a7157cd6-7ddd-4c57-9953-31e112d28409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:48.514212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 946/1893] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:48.518326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeb191-7a10-4090-a7c3-e94d97ce971f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:48.528796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efeb191-7a10-4090-a7c3-e94d97ce971f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:48.533316 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:48.533911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.534190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.534462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.535100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:48.535461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:48.537081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:48.548745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 948/1894] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:28:48.552866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7157cd6-7ddd-4c57-9953-31e112d28409 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:48.562521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7157cd6-7ddd-4c57-9953-31e112d28409 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:48.567247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87671f4c-6c39-49f6-989a-c8da9f8c39bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:48.567901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87671f4c-6c39-49f6-989a-c8da9f8c39bd admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.568207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87671f4c-6c39-49f6-989a-c8da9f8c39bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.569075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87671f4c-6c39-49f6-989a-c8da9f8c39bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.569828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87671f4c-6c39-49f6-989a-c8da9f8c39bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:48.571714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87671f4c-6c39-49f6-989a-c8da9f8c39bd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:48.599352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 947/1895] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:48 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:48.603464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:48.613604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659e1a1-f0aa-4211-aac0-973f819978ba admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:48.670925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9888133-6e61-4d01-81a1-ea8d248280ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:48.671536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9888133-6e61-4d01-81a1-ea8d248280ec admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.671815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9888133-6e61-4d01-81a1-ea8d248280ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.672074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9888133-6e61-4d01-81a1-ea8d248280ec admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.672742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9888133-6e61-4d01-81a1-ea8d248280ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:48.674669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9888133-6e61-4d01-81a1-ea8d248280ec admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:48.698989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 949/1896] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:48 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:48.703083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87671f4c-6c39-49f6-989a-c8da9f8c39bd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:48.714296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87671f4c-6c39-49f6-989a-c8da9f8c39bd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:48.718926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff317b1a-b4ba-4d17-82ac-32d38e06af40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:48.719543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317b1a-b4ba-4d17-82ac-32d38e06af40 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.719823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317b1a-b4ba-4d17-82ac-32d38e06af40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.720091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff317b1a-b4ba-4d17-82ac-32d38e06af40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.720762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff317b1a-b4ba-4d17-82ac-32d38e06af40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:48.722331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff317b1a-b4ba-4d17-82ac-32d38e06af40 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:48.733049 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 948/1897] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:48.737122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9888133-6e61-4d01-81a1-ea8d248280ec admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:48.747708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9888133-6e61-4d01-81a1-ea8d248280ec admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:48.753237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c16e1-8b7a-44e4-baf6-177a5b2564ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:48.753841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c16e1-8b7a-44e4-baf6-177a5b2564ab admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.754121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c16e1-8b7a-44e4-baf6-177a5b2564ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.754391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c16e1-8b7a-44e4-baf6-177a5b2564ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.755045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c16e1-8b7a-44e4-baf6-177a5b2564ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:48.757253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c16e1-8b7a-44e4-baf6-177a5b2564ab admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:48.768028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 950/1898] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:48.772222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff317b1a-b4ba-4d17-82ac-32d38e06af40 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:48.782207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff317b1a-b4ba-4d17-82ac-32d38e06af40 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:48.788353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc83310-b4c7-4203-b7b0-b9514170d504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:48.788832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc83310-b4c7-4203-b7b0-b9514170d504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.789179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc83310-b4c7-4203-b7b0-b9514170d504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.789524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc83310-b4c7-4203-b7b0-b9514170d504 admin admin] PATH_INFO: `/v3/projects/93ffc7af281a4b0e98cb9fd642768881/users/ac1f3aac7106414cadd8d242ede67e6b/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.812725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc83310-b4c7-4203-b7b0-b9514170d504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ffc7af281a4b0e98cb9fd642768881, user_id=ac1f3aac7106414cadd8d242ede67e6b, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:48.815370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc83310-b4c7-4203-b7b0-b9514170d504 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:48.831336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 949/1899] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:48 2022] PUT /identity/v3/projects/93ffc7af281a4b0e98cb9fd642768881/users/ac1f3aac7106414cadd8d242ede67e6b/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:48.841396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8404-0fe0-4cc2-bf16-3f2571b7be98 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:48.841702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8404-0fe0-4cc2-bf16-3f2571b7be98 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:48.841995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc8404-0fe0-4cc2-bf16-3f2571b7be98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:48.869797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c2dc8404-0fe0-4cc2-bf16-3f2571b7be98 None None] MFA Rules not processed for user `ac1f3aac7106414cadd8d242ede67e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:48.870925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c2dc8404-0fe0-4cc2-bf16-3f2571b7be98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:48.938457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 951/1900] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:48.943052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc83310-b4c7-4203-b7b0-b9514170d504 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:49.009363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc83310-b4c7-4203-b7b0-b9514170d504 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:49.013989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781236-c5b7-40ce-8c42-f1223742aa1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:49.014594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781236-c5b7-40ce-8c42-f1223742aa1a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.014870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781236-c5b7-40ce-8c42-f1223742aa1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.015139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781236-c5b7-40ce-8c42-f1223742aa1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.015759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c781236-c5b7-40ce-8c42-f1223742aa1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:49.017671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c781236-c5b7-40ce-8c42-f1223742aa1a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:49.049240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8404-0fe0-4cc2-bf16-3f2571b7be98 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:49.049759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 950/1901] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:48 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:49.061889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc8404-0fe0-4cc2-bf16-3f2571b7be98 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:49.113013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:49.129671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97154a-382e-4427-b777-652ff5e449ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:49.130213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97154a-382e-4427-b777-652ff5e449ab admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.130487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97154a-382e-4427-b777-652ff5e449ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.130751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97154a-382e-4427-b777-652ff5e449ab admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.131618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97154a-382e-4427-b777-652ff5e449ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:49.133575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97154a-382e-4427-b777-652ff5e449ab admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:49.153808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 952/1902] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:49 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:28:49.157813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781236-c5b7-40ce-8c42-f1223742aa1a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:49.169084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781236-c5b7-40ce-8c42-f1223742aa1a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:49.173956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa3690-83aa-42e4-b732-39ed712be527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:49.174555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3690-83aa-42e4-b732-39ed712be527 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.174833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3690-83aa-42e4-b732-39ed712be527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.175469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3690-83aa-42e4-b732-39ed712be527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.175739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa3690-83aa-42e4-b732-39ed712be527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:49.177339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa3690-83aa-42e4-b732-39ed712be527 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:49.188985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 951/1903] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:49.193195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97154a-382e-4427-b777-652ff5e449ab admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:49.203378 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97154a-382e-4427-b777-652ff5e449ab admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:49.208042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2470-f69d-44f6-afcc-fb18412eb4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:49.208824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2470-f69d-44f6-afcc-fb18412eb4c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.209215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2470-f69d-44f6-afcc-fb18412eb4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.209549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2470-f69d-44f6-afcc-fb18412eb4c7 admin admin] PATH_INFO: `/v3/projects/d91bb274604e430584541fa9e37cb692/users/343b77bd93da47518701d7dea84921e0/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.226902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a2470-f69d-44f6-afcc-fb18412eb4c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91bb274604e430584541fa9e37cb692, user_id=343b77bd93da47518701d7dea84921e0, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:49.230119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a2470-f69d-44f6-afcc-fb18412eb4c7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:49.244748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 953/1904] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:49 2022] PUT /identity/v3/projects/d91bb274604e430584541fa9e37cb692/users/343b77bd93da47518701d7dea84921e0/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:49.248563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa3690-83aa-42e4-b732-39ed712be527 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:49.258904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa3690-83aa-42e4-b732-39ed712be527 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:49.307805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:49.323012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b845984-6f3b-4c5e-aff2-01da143b29a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:49.323662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b845984-6f3b-4c5e-aff2-01da143b29a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.323944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b845984-6f3b-4c5e-aff2-01da143b29a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.324310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b845984-6f3b-4c5e-aff2-01da143b29a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.324946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b845984-6f3b-4c5e-aff2-01da143b29a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:49.326541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b845984-6f3b-4c5e-aff2-01da143b29a7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:49.337743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 952/1905] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:49.342005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2470-f69d-44f6-afcc-fb18412eb4c7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:49.353346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2470-f69d-44f6-afcc-fb18412eb4c7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:49.358015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:49.358654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.358934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.359229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] PATH_INFO: `/v3/projects/d91bb274604e430584541fa9e37cb692/users/343b77bd93da47518701d7dea84921e0/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.361189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91bb274604e430584541fa9e37cb692, user_id=343b77bd93da47518701d7dea84921e0, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:49.363803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:49.373942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 954/1906] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:49 2022] PUT /identity/v3/projects/d91bb274604e430584541fa9e37cb692/users/343b77bd93da47518701d7dea84921e0/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:49.384316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644552d-c5a8-4cee-8f51-a60dfc17b5fd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.384534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644552d-c5a8-4cee-8f51-a60dfc17b5fd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.384828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644552d-c5a8-4cee-8f51-a60dfc17b5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.413223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-a644552d-c5a8-4cee-8f51-a60dfc17b5fd None None] MFA Rules not processed for user `343b77bd93da47518701d7dea84921e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:49.414544 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a644552d-c5a8-4cee-8f51-a60dfc17b5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:49.488090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 953/1907] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:49.499451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:49.501282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:49.575512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee631b-c557-47a4-98c7-8aeb720a417f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:49.579857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e44db-4c3e-4372-b2c3-f1a2f751bb49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:49.580582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e44db-4c3e-4372-b2c3-f1a2f751bb49 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.581428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e44db-4c3e-4372-b2c3-f1a2f751bb49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.581780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e44db-4c3e-4372-b2c3-f1a2f751bb49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.592567 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e44db-4c3e-4372-b2c3-f1a2f751bb49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:49.594864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e44db-4c3e-4372-b2c3-f1a2f751bb49 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:49.612012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 955/1908] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:49.665880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644552d-c5a8-4cee-8f51-a60dfc17b5fd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:49.667384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a644552d-c5a8-4cee-8f51-a60dfc17b5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:49.679225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644552d-c5a8-4cee-8f51-a60dfc17b5fd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:49.683090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195924-acb4-43ac-b552-40f3c2c5d67a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:49.683685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195924-acb4-43ac-b552-40f3c2c5d67a service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:49.683966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195924-acb4-43ac-b552-40f3c2c5d67a service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:49.684272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195924-acb4-43ac-b552-40f3c2c5d67a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:49.740651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195924-acb4-43ac-b552-40f3c2c5d67a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:49.742269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195924-acb4-43ac-b552-40f3c2c5d67a service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:49.750947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 954/1909] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:51.309610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e44db-4c3e-4372-b2c3-f1a2f751bb49 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:51.335659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e44db-4c3e-4372-b2c3-f1a2f751bb49 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:51.347635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6728492f-4890-47de-acd2-01aeab7d6f29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:51.348391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728492f-4890-47de-acd2-01aeab7d6f29 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:51.348668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728492f-4890-47de-acd2-01aeab7d6f29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:51.349024 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6728492f-4890-47de-acd2-01aeab7d6f29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:51.353768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6728492f-4890-47de-acd2-01aeab7d6f29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:51.533950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728492f-4890-47de-acd2-01aeab7d6f29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:51.535958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6728492f-4890-47de-acd2-01aeab7d6f29 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:51.565855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 956/1910] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:53.683736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195924-acb4-43ac-b552-40f3c2c5d67a service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:53.757192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195924-acb4-43ac-b552-40f3c2c5d67a service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:53.761900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:53.762531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:53.762807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:53.763077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] PATH_INFO: `/v3/users/ac1f3aac7106414cadd8d242ede67e6b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:53.764224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1f3aac7106414cadd8d242ede67e6b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:53.766141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:53.771735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:53.811793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1f3aac7106414cadd8d242ede67e6b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:53.821232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1f3aac7106414cadd8d242ede67e6b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:53.821900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1f3aac7106414cadd8d242ede67e6b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:53.834274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1f3aac7106414cadd8d242ede67e6b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:53.835557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1f3aac7106414cadd8d242ede67e6b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:53.847868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 955/1911] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:53 2022] DELETE /identity/v3/users/ac1f3aac7106414cadd8d242ede67e6b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:53.852819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f519b5-4501-4937-a08d-d64783009886 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:53.853272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f519b5-4501-4937-a08d-d64783009886 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:53.853357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f519b5-4501-4937-a08d-d64783009886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:53.872141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-26f519b5-4501-4937-a08d-d64783009886 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:53.873307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-26f519b5-4501-4937-a08d-d64783009886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:53.909697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:53.931663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 957/1912] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:53.941177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:53.942589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:53.998299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2720ec6-1c71-4219-82ae-9bf8c4bdb684 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:54.001919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad34ec-2dc1-4af0-bf2d-2bb1d9210a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:54.002521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad34ec-2dc1-4af0-bf2d-2bb1d9210a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:54.002799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad34ec-2dc1-4af0-bf2d-2bb1d9210a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:54.003070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad34ec-2dc1-4af0-bf2d-2bb1d9210a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:54.012418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad34ec-2dc1-4af0-bf2d-2bb1d9210a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:54.013900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad34ec-2dc1-4af0-bf2d-2bb1d9210a39 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:54.024218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 956/1913] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:54.432464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f519b5-4501-4937-a08d-d64783009886 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:54.433766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-26f519b5-4501-4937-a08d-d64783009886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:54.447033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f519b5-4501-4937-a08d-d64783009886 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:54.452831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:54.453048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:54.453393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:54.453788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] PATH_INFO: `/v3/projects/93ffc7af281a4b0e98cb9fd642768881` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:54.455188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ffc7af281a4b0e98cb9fd642768881)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:54.470586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:54.529508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ffc7af281a4b0e98cb9fd642768881', 'request_id': 'req-8f539d77-915f-4e37-8279-9644e2d2e25e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.536425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ffc7af281a4b0e98cb9fd642768881', 'request_id': 'req-8f539d77-915f-4e37-8279-9644e2d2e25e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.542996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ffc7af281a4b0e98cb9fd642768881', 'request_id': 'req-8f539d77-915f-4e37-8279-9644e2d2e25e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.558778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] The token cache is being invalidate because project 93ffc7af281a4b0e98cb9fd642768881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:54.559180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.559814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.561377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 958/1914] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:54 2022] DELETE /identity/v3/projects/93ffc7af281a4b0e98cb9fd642768881 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:54.565431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad34ec-2dc1-4af0-bf2d-2bb1d9210a39 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:54.576116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad34ec-2dc1-4af0-bf2d-2bb1d9210a39 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:54.633540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:54.634084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:54.634369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:54.634763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] PATH_INFO: `/v3/users/343b77bd93da47518701d7dea84921e0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:54.635907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=343b77bd93da47518701d7dea84921e0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:54.637870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:54.670088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:54.685471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343b77bd93da47518701d7dea84921e0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.692479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343b77bd93da47518701d7dea84921e0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.693231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343b77bd93da47518701d7dea84921e0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.706082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343b77bd93da47518701d7dea84921e0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.706752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-651df6e8-70f0-4e15-ac99-f153d68d30ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343b77bd93da47518701d7dea84921e0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:54.718631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 957/1915] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:54 2022] DELETE /identity/v3/users/343b77bd93da47518701d7dea84921e0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:55.058756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:55.060714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:55.073412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f539d77-915f-4e37-8279-9644e2d2e25e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:55.129705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:55.130461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.130787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.131059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] PATH_INFO: `/v3/projects/d91bb274604e430584541fa9e37cb692` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.132228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91bb274604e430584541fa9e37cb692)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:55.134137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:55.180296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91bb274604e430584541fa9e37cb692', 'request_id': 'req-9bb5619d-d6d0-4f50-986e-d47aadc42b29'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:55.186555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91bb274604e430584541fa9e37cb692', 'request_id': 'req-9bb5619d-d6d0-4f50-986e-d47aadc42b29'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:55.192310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91bb274604e430584541fa9e37cb692', 'request_id': 'req-9bb5619d-d6d0-4f50-986e-d47aadc42b29'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:55.195877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:55.206742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] The token cache is being invalidate because project d91bb274604e430584541fa9e37cb692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:28:55.207195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:55.207823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9bb5619d-d6d0-4f50-986e-d47aadc42b29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:55.209412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 959/1916] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:55 2022] DELETE /identity/v3/projects/d91bb274604e430584541fa9e37cb692 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:55.214256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1e83d-03ba-4c11-9cfa-fd8fb095f048 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.214539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1e83d-03ba-4c11-9cfa-fd8fb095f048 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.214811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1e83d-03ba-4c11-9cfa-fd8fb095f048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.234673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-25a1e83d-03ba-4c11-9cfa-fd8fb095f048 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:55.236042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-25a1e83d-03ba-4c11-9cfa-fd8fb095f048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:55.297849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 958/1917] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:55.305353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2a455-02a9-49e0-9770-f05e6a58c865 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.305646 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2a455-02a9-49e0-9770-f05e6a58c865 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.305921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2a455-02a9-49e0-9770-f05e6a58c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.326669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-09c2a455-02a9-49e0-9770-f05e6a58c865 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:55.327895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-09c2a455-02a9-49e0-9770-f05e6a58c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:55.336638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 960/1918] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:55.340973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1e83d-03ba-4c11-9cfa-fd8fb095f048 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:55.351231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1e83d-03ba-4c11-9cfa-fd8fb095f048 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:55.355875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:55.356523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.356802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.357072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.357723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:28:55.358025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:55.359654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:55.373814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 959/1919] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:28:55.378213 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2a455-02a9-49e0-9770-f05e6a58c865 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:55.388423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2a455-02a9-49e0-9770-f05e6a58c865 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:55.393219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22993cb4-2da5-476a-b471-a66ce15fba10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:55.393822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22993cb4-2da5-476a-b471-a66ce15fba10 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.394100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22993cb4-2da5-476a-b471-a66ce15fba10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.394372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22993cb4-2da5-476a-b471-a66ce15fba10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.395048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22993cb4-2da5-476a-b471-a66ce15fba10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:55.397053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22993cb4-2da5-476a-b471-a66ce15fba10 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:55.434458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 961/1920] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:55 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:55.438534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:55.450402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22488ef8-02a7-4b77-9494-9feb3f5e2de3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:55.511813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19825a-17e0-442d-8e6d-b1c0c58e19eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:55.512453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19825a-17e0-442d-8e6d-b1c0c58e19eb admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.512729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19825a-17e0-442d-8e6d-b1c0c58e19eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.513031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19825a-17e0-442d-8e6d-b1c0c58e19eb admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.515012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19825a-17e0-442d-8e6d-b1c0c58e19eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:55.515592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19825a-17e0-442d-8e6d-b1c0c58e19eb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:55.538770 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 960/1921] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:55 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:55.542656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22993cb4-2da5-476a-b471-a66ce15fba10 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:55.552930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22993cb4-2da5-476a-b471-a66ce15fba10 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:55.559256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d10541-2b14-4cab-9775-e2ac80eb008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:55.560015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10541-2b14-4cab-9775-e2ac80eb008e admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.560432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10541-2b14-4cab-9775-e2ac80eb008e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.560899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d10541-2b14-4cab-9775-e2ac80eb008e admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.561699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d10541-2b14-4cab-9775-e2ac80eb008e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:55.563188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d10541-2b14-4cab-9775-e2ac80eb008e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:55.577063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 962/1922] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:55.580993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19825a-17e0-442d-8e6d-b1c0c58e19eb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:55.590951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19825a-17e0-442d-8e6d-b1c0c58e19eb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:55.595571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391659fe-72cc-4c40-a3c8-9705d71d006c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:55.596204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391659fe-72cc-4c40-a3c8-9705d71d006c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.596488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391659fe-72cc-4c40-a3c8-9705d71d006c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.596760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391659fe-72cc-4c40-a3c8-9705d71d006c admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.597399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391659fe-72cc-4c40-a3c8-9705d71d006c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:55.599016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391659fe-72cc-4c40-a3c8-9705d71d006c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:55.612352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 961/1923] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:55.614163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d10541-2b14-4cab-9775-e2ac80eb008e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:55.624804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d10541-2b14-4cab-9775-e2ac80eb008e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:55.629550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bc493-a6ec-4efd-930e-7cc6cc15bea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:55.630188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc493-a6ec-4efd-930e-7cc6cc15bea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.630465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc493-a6ec-4efd-930e-7cc6cc15bea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.630735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc493-a6ec-4efd-930e-7cc6cc15bea5 admin admin] PATH_INFO: `/v3/projects/9ff68166c3af41e2b37f3b761a1ca017/users/17ea91493e324fb99cddaa0f121ad36c/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.648354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79bc493-a6ec-4efd-930e-7cc6cc15bea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff68166c3af41e2b37f3b761a1ca017, user_id=17ea91493e324fb99cddaa0f121ad36c, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:55.650995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79bc493-a6ec-4efd-930e-7cc6cc15bea5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:55.664635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 963/1924] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:55 2022] PUT /identity/v3/projects/9ff68166c3af41e2b37f3b761a1ca017/users/17ea91493e324fb99cddaa0f121ad36c/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:55.676800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05d1ec-2ebc-434e-8092-d5d61a2681cb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.677117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05d1ec-2ebc-434e-8092-d5d61a2681cb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.677393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05d1ec-2ebc-434e-8092-d5d61a2681cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.704783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:55.708578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-eb05d1ec-2ebc-434e-8092-d5d61a2681cb None None] MFA Rules not processed for user `17ea91493e324fb99cddaa0f121ad36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:55.712417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-eb05d1ec-2ebc-434e-8092-d5d61a2681cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:55.802191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 962/1925] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:55.809288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bc493-a6ec-4efd-930e-7cc6cc15bea5 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:55.934494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bc493-a6ec-4efd-930e-7cc6cc15bea5 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:55.946724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a052-ef6b-4064-9d58-f7dd68d6fee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:55.947358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a052-ef6b-4064-9d58-f7dd68d6fee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:55.947637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a052-ef6b-4064-9d58-f7dd68d6fee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:55.947902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59a052-ef6b-4064-9d58-f7dd68d6fee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:55.948618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59a052-ef6b-4064-9d58-f7dd68d6fee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:55.954311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59a052-ef6b-4064-9d58-f7dd68d6fee5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:55.998277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 964/1926] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:55 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:56.002986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05d1ec-2ebc-434e-8092-d5d61a2681cb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.013872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05d1ec-2ebc-434e-8092-d5d61a2681cb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.082160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6116a-048f-4340-8e46-26d8dbe35d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.082837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6116a-048f-4340-8e46-26d8dbe35d31 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.083183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6116a-048f-4340-8e46-26d8dbe35d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.083519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6116a-048f-4340-8e46-26d8dbe35d31 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.084803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6116a-048f-4340-8e46-26d8dbe35d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.087205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6116a-048f-4340-8e46-26d8dbe35d31 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.111242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 963/1927] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:56 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:56.115191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a052-ef6b-4064-9d58-f7dd68d6fee5 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.133014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59a052-ef6b-4064-9d58-f7dd68d6fee5 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.138258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8142d-418a-4cd6-8490-a55b28c43f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.138868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8142d-418a-4cd6-8490-a55b28c43f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.139145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8142d-418a-4cd6-8490-a55b28c43f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.139410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8142d-418a-4cd6-8490-a55b28c43f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.140050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8142d-418a-4cd6-8490-a55b28c43f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.141829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8142d-418a-4cd6-8490-a55b28c43f8f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.156523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 965/1928] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:56.160867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6116a-048f-4340-8e46-26d8dbe35d31 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.174246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6116a-048f-4340-8e46-26d8dbe35d31 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.179501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1d9ad-d38f-45e7-9285-ab5c749695d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.180152 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1d9ad-d38f-45e7-9285-ab5c749695d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.180459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1d9ad-d38f-45e7-9285-ab5c749695d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.180725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1d9ad-d38f-45e7-9285-ab5c749695d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.181620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1d9ad-d38f-45e7-9285-ab5c749695d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.183220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1d9ad-d38f-45e7-9285-ab5c749695d9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.193226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 964/1929] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:56.197654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8142d-418a-4cd6-8490-a55b28c43f8f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.207817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8142d-418a-4cd6-8490-a55b28c43f8f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.212934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076e23a-b609-4ced-b4be-b97dee9dd6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.213560 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076e23a-b609-4ced-b4be-b97dee9dd6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.213840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076e23a-b609-4ced-b4be-b97dee9dd6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.214114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076e23a-b609-4ced-b4be-b97dee9dd6ae admin admin] PATH_INFO: `/v3/projects/56ff09483ec8431b8f0e1902a08b33d8/users/7e10e392f53b4cd597ea8c5c433ccd8c/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.228812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076e23a-b609-4ced-b4be-b97dee9dd6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ff09483ec8431b8f0e1902a08b33d8, user_id=7e10e392f53b4cd597ea8c5c433ccd8c, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.231456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8076e23a-b609-4ced-b4be-b97dee9dd6ae admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.244016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 966/1930] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:56 2022] PUT /identity/v3/projects/56ff09483ec8431b8f0e1902a08b33d8/users/7e10e392f53b4cd597ea8c5c433ccd8c/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:56.254672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5ba19-1281-44a1-ad7c-4e0c1d18275b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.255172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5ba19-1281-44a1-ad7c-4e0c1d18275b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.255547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5ba19-1281-44a1-ad7c-4e0c1d18275b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.286255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-12c5ba19-1281-44a1-ad7c-4e0c1d18275b None None] MFA Rules not processed for user `7e10e392f53b4cd597ea8c5c433ccd8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:56.288262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-12c5ba19-1281-44a1-ad7c-4e0c1d18275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:56.355053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 965/1931] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:28:56.359499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076e23a-b609-4ced-b4be-b97dee9dd6ae admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.406812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:56.431409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076e23a-b609-4ced-b4be-b97dee9dd6ae admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.437240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef5531-8dd9-4f87-b52f-679a246dfc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.437855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5531-8dd9-4f87-b52f-679a246dfc21 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.438210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5531-8dd9-4f87-b52f-679a246dfc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.438397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5531-8dd9-4f87-b52f-679a246dfc21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.439726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef5531-8dd9-4f87-b52f-679a246dfc21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.441136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef5531-8dd9-4f87-b52f-679a246dfc21 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.471030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 967/1932] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:28:56 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:56.476406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5ba19-1281-44a1-ad7c-4e0c1d18275b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.488001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5ba19-1281-44a1-ad7c-4e0c1d18275b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.498714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:56.620743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182bd4-f7e9-48f7-9080-8dc17feec339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.621327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182bd4-f7e9-48f7-9080-8dc17feec339 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.621739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182bd4-f7e9-48f7-9080-8dc17feec339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.622080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5182bd4-f7e9-48f7-9080-8dc17feec339 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.622802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5182bd4-f7e9-48f7-9080-8dc17feec339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.625190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5182bd4-f7e9-48f7-9080-8dc17feec339 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.682276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 966/1933] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:28:56 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:28:56.686278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef5531-8dd9-4f87-b52f-679a246dfc21 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.708535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef5531-8dd9-4f87-b52f-679a246dfc21 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.716465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec23d87-a484-41eb-813f-a1e7a03d005a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.717092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec23d87-a484-41eb-813f-a1e7a03d005a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.717399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec23d87-a484-41eb-813f-a1e7a03d005a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.717642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec23d87-a484-41eb-813f-a1e7a03d005a admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.718298 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec23d87-a484-41eb-813f-a1e7a03d005a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.720085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec23d87-a484-41eb-813f-a1e7a03d005a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.754083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182bd4-f7e9-48f7-9080-8dc17feec339 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.763925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 968/1934] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:56.766072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5182bd4-f7e9-48f7-9080-8dc17feec339 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.780493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd903bb-f7d5-4b80-b1b4-978c993719b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.781217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd903bb-f7d5-4b80-b1b4-978c993719b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.781632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd903bb-f7d5-4b80-b1b4-978c993719b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.781998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd903bb-f7d5-4b80-b1b4-978c993719b7 admin admin] PATH_INFO: `/v3/projects/c1769ef0ec554ac2aabb536ac158be96/users/ad3ff77757ad49f68271c3ad918157b0/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.805610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd903bb-f7d5-4b80-b1b4-978c993719b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1769ef0ec554ac2aabb536ac158be96, user_id=ad3ff77757ad49f68271c3ad918157b0, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.808592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd903bb-f7d5-4b80-b1b4-978c993719b7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.827045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 967/1935] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:56 2022] PUT /identity/v3/projects/c1769ef0ec554ac2aabb536ac158be96/users/ad3ff77757ad49f68271c3ad918157b0/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:28:56.831388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec23d87-a484-41eb-813f-a1e7a03d005a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.843115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec23d87-a484-41eb-813f-a1e7a03d005a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.910077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d28941-ebc0-445c-ba08-31465ec51b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.910660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d28941-ebc0-445c-ba08-31465ec51b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.910952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d28941-ebc0-445c-ba08-31465ec51b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.911230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d28941-ebc0-445c-ba08-31465ec51b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.911880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d28941-ebc0-445c-ba08-31465ec51b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.913617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d28941-ebc0-445c-ba08-31465ec51b0f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.924575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 969/1936] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:28:56.930437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd903bb-f7d5-4b80-b1b4-978c993719b7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:56.944583 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd903bb-f7d5-4b80-b1b4-978c993719b7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:56.951349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:56.952179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.952524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.952900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] PATH_INFO: `/v3/projects/c1769ef0ec554ac2aabb536ac158be96/users/ad3ff77757ad49f68271c3ad918157b0/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:56.955352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1769ef0ec554ac2aabb536ac158be96, user_id=ad3ff77757ad49f68271c3ad918157b0, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:56.958214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:56.970059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 968/1937] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:28:56 2022] PUT /identity/v3/projects/c1769ef0ec554ac2aabb536ac158be96/users/ad3ff77757ad49f68271c3ad918157b0/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:56.994666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9bca8-27c3-4b7c-a8c8-e997964f214e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:56.995589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9bca8-27c3-4b7c-a8c8-e997964f214e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:56.995875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c9bca8-27c3-4b7c-a8c8-e997964f214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:57.031935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-50c9bca8-27c3-4b7c-a8c8-e997964f214e None None] MFA Rules not processed for user `ad3ff77757ad49f68271c3ad918157b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:57.033452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-50c9bca8-27c3-4b7c-a8c8-e997964f214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:57.102086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 970/1938] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:57.112767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:57.114438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:57.172384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822d8bb-d5a7-40f8-883c-7cbac0abd4b9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:57.175946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:57.176577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:57.179061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:57.179061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:57.189518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:57.191074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:57.203280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 969/1939] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:57.901798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9bca8-27c3-4b7c-a8c8-e997964f214e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:57.904038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-50c9bca8-27c3-4b7c-a8c8-e997964f214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:57.915951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c9bca8-27c3-4b7c-a8c8-e997964f214e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:57.920746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff500f-0e17-4da5-bd67-6492390b0bd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:57.921519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff500f-0e17-4da5-bd67-6492390b0bd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:57.921884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff500f-0e17-4da5-bd67-6492390b0bd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:57.922234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff500f-0e17-4da5-bd67-6492390b0bd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:57.968219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:58.009979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff500f-0e17-4da5-bd67-6492390b0bd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:58.011571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff500f-0e17-4da5-bd67-6492390b0bd3 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:58.021553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 971/1940] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:59.081044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:59.082499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:59.152910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a0b13-948e-4d5a-95aa-5cdf4361a786 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:59.158115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:59.158751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:59.159167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:59.159446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] PATH_INFO: `/v3/users/267cd1bf56534eef92049c43d68bfc5e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:59.160612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] RBAC: Authorizing `identity:delete_user(user_id=267cd1bf56534eef92049c43d68bfc5e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:59.163394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:59.169407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:28:59.213050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '267cd1bf56534eef92049c43d68bfc5e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:59.221110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '267cd1bf56534eef92049c43d68bfc5e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:59.221765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '267cd1bf56534eef92049c43d68bfc5e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:59.242630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '267cd1bf56534eef92049c43d68bfc5e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:59.243380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '267cd1bf56534eef92049c43d68bfc5e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:28:59.255381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 970/1941] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:28:59 2022] DELETE /identity/v3/users/267cd1bf56534eef92049c43d68bfc5e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:28:59.259937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16026a-ea92-4d15-8560-239cb9be1fd2 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:59.260455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16026a-ea92-4d15-8560-239cb9be1fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:59.260572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16026a-ea92-4d15-8560-239cb9be1fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:59.280307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-6d16026a-ea92-4d15-8560-239cb9be1fd2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:28:59.281693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6d16026a-ea92-4d15-8560-239cb9be1fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:59.356004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 972/1942] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:28:59.362744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:59.364257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:59.425466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d751651-c75a-4e96-a8fa-7597b0d4748d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:59.429728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce697-a2d8-446f-9a31-35c08557017d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:59.430346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce697-a2d8-446f-9a31-35c08557017d service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:59.430623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce697-a2d8-446f-9a31-35c08557017d service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:59.430892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce697-a2d8-446f-9a31-35c08557017d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:59.440277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ce697-a2d8-446f-9a31-35c08557017d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:59.441696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ce697-a2d8-446f-9a31-35c08557017d service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:28:59.452449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 971/1943] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:28:59.953144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16026a-ea92-4d15-8560-239cb9be1fd2 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:28:59.957296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6d16026a-ea92-4d15-8560-239cb9be1fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:28:59.981501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16026a-ea92-4d15-8560-239cb9be1fd2 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:28:59.988976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:28:59.989695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:28:59.990196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:28:59.990632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] PATH_INFO: `/v3/projects/614cc401a97144139a50f76b9b243bc9` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:28:59.992183 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=614cc401a97144139a50f76b9b243bc9)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:28:59.994314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:00.065103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614cc401a97144139a50f76b9b243bc9', 'request_id': 'req-d412195f-934a-44b0-8a4c-b58df5f31edc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.073610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614cc401a97144139a50f76b9b243bc9', 'request_id': 'req-d412195f-934a-44b0-8a4c-b58df5f31edc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.079495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614cc401a97144139a50f76b9b243bc9', 'request_id': 'req-d412195f-934a-44b0-8a4c-b58df5f31edc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.093728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] The token cache is being invalidate because project 614cc401a97144139a50f76b9b243bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:00.094020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.094645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.096218 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 973/1944] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:28:59 2022] DELETE /identity/v3/projects/614cc401a97144139a50f76b9b243bc9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:00.101841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce697-a2d8-446f-9a31-35c08557017d service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:00.112389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce697-a2d8-446f-9a31-35c08557017d service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:00.179400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:00.180287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:00.180692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:00.181491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] PATH_INFO: `/v3/users/b8448e0de42d49e69ca72628d85c3934` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:00.182634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8448e0de42d49e69ca72628d85c3934)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:00.184834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:00.226477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:00.246744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8448e0de42d49e69ca72628d85c3934'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.258109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8448e0de42d49e69ca72628d85c3934'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.258843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8448e0de42d49e69ca72628d85c3934'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.275153 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8448e0de42d49e69ca72628d85c3934'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.275894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f126b5e1-e9da-40bb-9ee7-1aceef32f7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8448e0de42d49e69ca72628d85c3934'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.290095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 972/1945] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:00 2022] DELETE /identity/v3/users/b8448e0de42d49e69ca72628d85c3934 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:00.594852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:00.596697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:00.618847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412195f-934a-44b0-8a4c-b58df5f31edc admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:00.712329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:00.713045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:00.713467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:00.713942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] PATH_INFO: `/v3/projects/3c4ef198141342ba9e19d46cbf10f9ad` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:00.715325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c4ef198141342ba9e19d46cbf10f9ad)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:00.717441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:00.720368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:00.778600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c4ef198141342ba9e19d46cbf10f9ad', 'request_id': 'req-6725979d-6784-43c6-8b95-3ef017e4c11b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.785578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c4ef198141342ba9e19d46cbf10f9ad', 'request_id': 'req-6725979d-6784-43c6-8b95-3ef017e4c11b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.793126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c4ef198141342ba9e19d46cbf10f9ad', 'request_id': 'req-6725979d-6784-43c6-8b95-3ef017e4c11b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.808969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] The token cache is being invalidate because project 3c4ef198141342ba9e19d46cbf10f9ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:00.809264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.809930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6725979d-6784-43c6-8b95-3ef017e4c11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:00.811746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 974/1946] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:00 2022] DELETE /identity/v3/projects/3c4ef198141342ba9e19d46cbf10f9ad => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:00.817752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70925c63-92e2-4acb-8cb3-6213b558e1c3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:00.818121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70925c63-92e2-4acb-8cb3-6213b558e1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:00.819102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70925c63-92e2-4acb-8cb3-6213b558e1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:00.842725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-70925c63-92e2-4acb-8cb3-6213b558e1c3 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:00.843933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-70925c63-92e2-4acb-8cb3-6213b558e1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:00.922675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 973/1947] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:00.931671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07e510-2eda-4948-96fb-79670d0df357 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:00.932052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07e510-2eda-4948-96fb-79670d0df357 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:00.932239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07e510-2eda-4948-96fb-79670d0df357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:00.945455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70925c63-92e2-4acb-8cb3-6213b558e1c3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:00.947077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-70925c63-92e2-4acb-8cb3-6213b558e1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:00.956054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1f07e510-2eda-4948-96fb-79670d0df357 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:00.957225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1f07e510-2eda-4948-96fb-79670d0df357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:00.960013 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70925c63-92e2-4acb-8cb3-6213b558e1c3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:00.964839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:00.965486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:00.965762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:00.966028 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] PATH_INFO: `/v3/users/2d7e0ddf97014038a53f33833acfc05f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:00.966804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 975/1948] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:00.968797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7e0ddf97014038a53f33833acfc05f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:00.970514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07e510-2eda-4948-96fb-79670d0df357 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:00.973429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:00.981468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07e510-2eda-4948-96fb-79670d0df357 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:00.988442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:00.989169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:00.989544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:00.989933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:00.991488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:00.991886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:00.993811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:01.012240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 976/1949] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:29:01.014271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:01.030898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf84f6-f26d-4379-a25a-60a1e716592b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:01.049688 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:01.052559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.053650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.053952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.064392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:01.066802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:01.069074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7e0ddf97014038a53f33833acfc05f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:01.081826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7e0ddf97014038a53f33833acfc05f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:01.083168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7e0ddf97014038a53f33833acfc05f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:01.107303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7e0ddf97014038a53f33833acfc05f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:01.108586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0dcc8033-2814-42a7-a760-46169b960971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7e0ddf97014038a53f33833acfc05f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:01.127700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 977/1950] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:01 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:01.128627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:01.133633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 974/1951] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:00 2022] DELETE /identity/v3/users/2d7e0ddf97014038a53f33833acfc05f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:01.135724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:01.137866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea876d-38f6-43de-98de-79185ca06b88 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.138217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea876d-38f6-43de-98de-79185ca06b88 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.138513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea876d-38f6-43de-98de-79185ca06b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.162069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-72ea876d-38f6-43de-98de-79185ca06b88 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:01.165357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72ea876d-38f6-43de-98de-79185ca06b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:01.186629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:01.250049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c0253-a696-4b76-b021-dc460bd0e033 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:01.276866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 975/1952] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:01.286925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea876d-38f6-43de-98de-79185ca06b88 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:01.289340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-72ea876d-38f6-43de-98de-79185ca06b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:01.328541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:01.351764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0175a-46ff-47e4-99c2-e45ab1ad952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:01.356362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0175a-46ff-47e4-99c2-e45ab1ad952d admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.356650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0175a-46ff-47e4-99c2-e45ab1ad952d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.356916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0175a-46ff-47e4-99c2-e45ab1ad952d admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.357623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0175a-46ff-47e4-99c2-e45ab1ad952d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:01.365180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0175a-46ff-47e4-99c2-e45ab1ad952d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:01.422067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea876d-38f6-43de-98de-79185ca06b88 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:01.427304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 978/1953] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:01 2022] POST /identity/v3/users => generated 289 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:01.427840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712f084-a876-4a4f-bea3-824bc59c5398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:01.428738 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712f084-a876-4a4f-bea3-824bc59c5398 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.429096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712f084-a876-4a4f-bea3-824bc59c5398 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.429940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712f084-a876-4a4f-bea3-824bc59c5398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.431941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0175a-46ff-47e4-99c2-e45ab1ad952d admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:01.443168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0175a-46ff-47e4-99c2-e45ab1ad952d admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:01.444634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712f084-a876-4a4f-bea3-824bc59c5398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:01.446274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712f084-a876-4a4f-bea3-824bc59c5398 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:01.449536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15806d5e-74c5-4bd1-be5c-e498ae6cfdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:01.450370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15806d5e-74c5-4bd1-be5c-e498ae6cfdaa admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.450812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15806d5e-74c5-4bd1-be5c-e498ae6cfdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.451282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15806d5e-74c5-4bd1-be5c-e498ae6cfdaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.452091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15806d5e-74c5-4bd1-be5c-e498ae6cfdaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:01.453858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15806d5e-74c5-4bd1-be5c-e498ae6cfdaa admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:01.459572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 976/1954] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:29:01.467439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 979/1955] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:01.471693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712f084-a876-4a4f-bea3-824bc59c5398 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:01.482434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712f084-a876-4a4f-bea3-824bc59c5398 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:01.487242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dffae-8fb8-4c4a-8f9e-14794da53b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:01.487926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dffae-8fb8-4c4a-8f9e-14794da53b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.488287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dffae-8fb8-4c4a-8f9e-14794da53b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.488624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dffae-8fb8-4c4a-8f9e-14794da53b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.489327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dffae-8fb8-4c4a-8f9e-14794da53b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:01.491073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dffae-8fb8-4c4a-8f9e-14794da53b4e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:01.503641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 977/1956] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:01.508086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15806d5e-74c5-4bd1-be5c-e498ae6cfdaa admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:01.518504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15806d5e-74c5-4bd1-be5c-e498ae6cfdaa admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:01.524544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:01.525197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.525480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.525751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] PATH_INFO: `/v3/projects/68eba302ba704d6eae134f3fe53935c4/users/742b13f45eb54d90a82b409111ff64f5/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.543145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68eba302ba704d6eae134f3fe53935c4, user_id=742b13f45eb54d90a82b409111ff64f5, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:01.545830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:01.559934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 980/1957] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:01 2022] PUT /identity/v3/projects/68eba302ba704d6eae134f3fe53935c4/users/742b13f45eb54d90a82b409111ff64f5/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:01.572834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece79bc-7c92-4c1e-84a3-391481a40ca3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.573331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece79bc-7c92-4c1e-84a3-391481a40ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.573530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece79bc-7c92-4c1e-84a3-391481a40ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.603889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8ece79bc-7c92-4c1e-84a3-391481a40ca3 None None] MFA Rules not processed for user `742b13f45eb54d90a82b409111ff64f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:01.605138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8ece79bc-7c92-4c1e-84a3-391481a40ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:01.671965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 978/1958] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:01.684434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:01.686591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:01.799304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c65d5-9394-4f8a-8bd2-990013177f57 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:01.805418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7e527-b0c4-4509-ad76-1add5efd75e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7a2237c18934cb2aa3699a5776874ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:01.805964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7e527-b0c4-4509-ad76-1add5efd75e3 service glance] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:01.806509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7e527-b0c4-4509-ad76-1add5efd75e3 service glance] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:01.807062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7e527-b0c4-4509-ad76-1add5efd75e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:01.825622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7e527-b0c4-4509-ad76-1add5efd75e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:01.827053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab7e527-b0c4-4509-ad76-1add5efd75e3 service glance] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:01.837940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 981/1959] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:01.974362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece79bc-7c92-4c1e-84a3-391481a40ca3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:01.976195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8ece79bc-7c92-4c1e-84a3-391481a40ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:02.040844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:02.069050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece79bc-7c92-4c1e-84a3-391481a40ca3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:02.075561 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:02.076257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:02.076540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:02.076811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] PATH_INFO: `/v3/projects/9b87a927ba4b4841b8e4db8c9dd6dbf4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:02.077967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b87a927ba4b4841b8e4db8c9dd6dbf4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:02.080044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:02.137856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b87a927ba4b4841b8e4db8c9dd6dbf4', 'request_id': 'req-94897a1c-92cf-4b70-a32f-720dbc4e75b8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.147231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b87a927ba4b4841b8e4db8c9dd6dbf4', 'request_id': 'req-94897a1c-92cf-4b70-a32f-720dbc4e75b8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.155063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b87a927ba4b4841b8e4db8c9dd6dbf4', 'request_id': 'req-94897a1c-92cf-4b70-a32f-720dbc4e75b8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.165603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] The token cache is being invalidate because project 9b87a927ba4b4841b8e4db8c9dd6dbf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:02.165892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.166556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.168050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 979/1960] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:01 2022] DELETE /identity/v3/projects/9b87a927ba4b4841b8e4db8c9dd6dbf4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:02.172580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7e527-b0c4-4509-ad76-1add5efd75e3 service glance] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:02.186360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7e527-b0c4-4509-ad76-1add5efd75e3 service glance] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:02.260753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:02.261410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:02.261693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:02.261969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] PATH_INFO: `/v3/users/8502badb6d2e4b9f97a5a3f1adf252aa` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:02.266471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8502badb6d2e4b9f97a5a3f1adf252aa)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:02.268634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:02.316555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:02.330649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8502badb6d2e4b9f97a5a3f1adf252aa'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.371159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8502badb6d2e4b9f97a5a3f1adf252aa'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.372039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8502badb6d2e4b9f97a5a3f1adf252aa'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.385274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8502badb6d2e4b9f97a5a3f1adf252aa'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.402895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8502badb6d2e4b9f97a5a3f1adf252aa'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:02.415672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 982/1961] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:02 2022] DELETE /identity/v3/users/8502badb6d2e4b9f97a5a3f1adf252aa => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:02.765634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:02.767631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:02.868509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94897a1c-92cf-4b70-a32f-720dbc4e75b8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:02.922377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6ec7e-8510-4d1b-9783-aecddd9a3b0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:02.923246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ec7e-8510-4d1b-9783-aecddd9a3b0b service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:02.923572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ec7e-8510-4d1b-9783-aecddd9a3b0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:02.923844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ec7e-8510-4d1b-9783-aecddd9a3b0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:02.926024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-11f6ec7e-8510-4d1b-9783-aecddd9a3b0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:03.043987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:03.065438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6ec7e-8510-4d1b-9783-aecddd9a3b0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:03.068865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6ec7e-8510-4d1b-9783-aecddd9a3b0b service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:03.087781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 980/1962] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:03.104632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:03.109504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:03.185306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb11e1-fb47-4f18-8267-2807db434924 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:03.190672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:03.191313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.191588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.191857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] PATH_INFO: `/v3/projects/373574a194b744a3b5cd359217b2c616` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.194203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] RBAC: Authorizing `identity:delete_project(project_id=373574a194b744a3b5cd359217b2c616)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:03.196153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:03.267522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '373574a194b744a3b5cd359217b2c616', 'request_id': 'req-873883bf-c815-49d0-aabd-b8df75c046de'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:03.278467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '373574a194b744a3b5cd359217b2c616', 'request_id': 'req-873883bf-c815-49d0-aabd-b8df75c046de'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:03.285914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '373574a194b744a3b5cd359217b2c616', 'request_id': 'req-873883bf-c815-49d0-aabd-b8df75c046de'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:03.301523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] The token cache is being invalidate because project 373574a194b744a3b5cd359217b2c616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:03.302357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:03.303102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-873883bf-c815-49d0-aabd-b8df75c046de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:03.304996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 983/1963] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:03 2022] DELETE /identity/v3/projects/373574a194b744a3b5cd359217b2c616 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:03.310860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363098a7-dfbe-407d-b023-acb39c326632 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.311387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363098a7-dfbe-407d-b023-acb39c326632 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.311475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363098a7-dfbe-407d-b023-acb39c326632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.334664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-363098a7-dfbe-407d-b023-acb39c326632 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:03.335932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-363098a7-dfbe-407d-b023-acb39c326632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:03.418167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 981/1964] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:03.424919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cba670-7591-458a-a645-f8998592a712 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.426305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cba670-7591-458a-a645-f8998592a712 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.426305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cba670-7591-458a-a645-f8998592a712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.450887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-03cba670-7591-458a-a645-f8998592a712 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:03.452653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-03cba670-7591-458a-a645-f8998592a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:03.463029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 984/1965] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:03.467339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363098a7-dfbe-407d-b023-acb39c326632 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:03.478719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363098a7-dfbe-407d-b023-acb39c326632 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:03.483462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:03.484082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.484388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.484658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.485312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:03.485610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:03.487341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:03.503049 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 982/1966] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:29:03.504206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cba670-7591-458a-a645-f8998592a712 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:03.513571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cba670-7591-458a-a645-f8998592a712 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:03.518292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b634c2-a69c-48f7-9532-e6b01fe7979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:03.518896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b634c2-a69c-48f7-9532-e6b01fe7979b admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.519171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b634c2-a69c-48f7-9532-e6b01fe7979b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.519445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b634c2-a69c-48f7-9532-e6b01fe7979b admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.520067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b634c2-a69c-48f7-9532-e6b01fe7979b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:03.522843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b634c2-a69c-48f7-9532-e6b01fe7979b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:03.561664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 985/1967] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:03 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:03.565537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:03.581122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5b9aa-4876-40ca-a260-fd017afdb2f3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:03.660868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e28a3-7f89-48bd-8ef4-f10f68356dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:03.661578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e28a3-7f89-48bd-8ef4-f10f68356dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.661971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e28a3-7f89-48bd-8ef4-f10f68356dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.663093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e28a3-7f89-48bd-8ef4-f10f68356dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.663843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e28a3-7f89-48bd-8ef4-f10f68356dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:03.665957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e28a3-7f89-48bd-8ef4-f10f68356dc3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:03.702471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 983/1968] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:03 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:03.707022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b634c2-a69c-48f7-9532-e6b01fe7979b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:03.718790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b634c2-a69c-48f7-9532-e6b01fe7979b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:03.723661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa72817-61ed-42ea-acd9-1da3beb596e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:03.724314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa72817-61ed-42ea-acd9-1da3beb596e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.725223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa72817-61ed-42ea-acd9-1da3beb596e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.725508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa72817-61ed-42ea-acd9-1da3beb596e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.726407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa72817-61ed-42ea-acd9-1da3beb596e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:03.728049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa72817-61ed-42ea-acd9-1da3beb596e9 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:03.749320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 986/1969] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:03.753230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e28a3-7f89-48bd-8ef4-f10f68356dc3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:03.765225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e28a3-7f89-48bd-8ef4-f10f68356dc3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:03.773180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92aadb-e2f9-4dd2-a8d6-79d367b1e25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:03.774097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92aadb-e2f9-4dd2-a8d6-79d367b1e25b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.774605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92aadb-e2f9-4dd2-a8d6-79d367b1e25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.775092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92aadb-e2f9-4dd2-a8d6-79d367b1e25b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.775966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92aadb-e2f9-4dd2-a8d6-79d367b1e25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:03.779042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92aadb-e2f9-4dd2-a8d6-79d367b1e25b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:03.797191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 984/1970] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:03.801165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa72817-61ed-42ea-acd9-1da3beb596e9 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:03.829159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:03.843265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa72817-61ed-42ea-acd9-1da3beb596e9 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:03.861275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:03.863621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.864013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.864468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] PATH_INFO: `/v3/projects/47bc0ace46a64934810bbc12e6c470d7/users/0a9fa4e50e7b432f9d48d16a50ae7fc0/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.904217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bc0ace46a64934810bbc12e6c470d7, user_id=0a9fa4e50e7b432f9d48d16a50ae7fc0, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:03.906964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:03.944853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 987/1971] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:03 2022] PUT /identity/v3/projects/47bc0ace46a64934810bbc12e6c470d7/users/0a9fa4e50e7b432f9d48d16a50ae7fc0/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:03.955396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b58f5-d958-4f45-9f10-6fb15dd1045d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:03.955682 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b58f5-d958-4f45-9f10-6fb15dd1045d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:03.955988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b58f5-d958-4f45-9f10-6fb15dd1045d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:03.994769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-006b58f5-d958-4f45-9f10-6fb15dd1045d None None] MFA Rules not processed for user `0a9fa4e50e7b432f9d48d16a50ae7fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:03.996147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-006b58f5-d958-4f45-9f10-6fb15dd1045d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:04.107195 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:04.146216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 985/1972] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:04.167641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:04.169460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:04.266294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2e903-2ebb-46ea-8213-c3bfaa47633c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:04.273868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56088d16-a94f-48cc-83dd-62f47ce8b4e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:04.274815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56088d16-a94f-48cc-83dd-62f47ce8b4e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:04.276270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56088d16-a94f-48cc-83dd-62f47ce8b4e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:04.276270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56088d16-a94f-48cc-83dd-62f47ce8b4e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:04.288346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56088d16-a94f-48cc-83dd-62f47ce8b4e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:04.290459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56088d16-a94f-48cc-83dd-62f47ce8b4e5 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:04.308196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 988/1973] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:04.329263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b58f5-d958-4f45-9f10-6fb15dd1045d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:04.407972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b58f5-d958-4f45-9f10-6fb15dd1045d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:04.413877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:04.414892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:04.414892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:04.415460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] PATH_INFO: `/v3/users/0a9fa4e50e7b432f9d48d16a50ae7fc0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:04.416740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9fa4e50e7b432f9d48d16a50ae7fc0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:04.419163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:04.478775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9fa4e50e7b432f9d48d16a50ae7fc0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:04.485952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9fa4e50e7b432f9d48d16a50ae7fc0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:04.486732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9fa4e50e7b432f9d48d16a50ae7fc0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:04.501214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9fa4e50e7b432f9d48d16a50ae7fc0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:04.502378 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9fa4e50e7b432f9d48d16a50ae7fc0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:04.517296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 986/1974] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:04 2022] DELETE /identity/v3/users/0a9fa4e50e7b432f9d48d16a50ae7fc0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:04.521545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ec13-84af-493b-848a-1363e34c137d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:04.522481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ec13-84af-493b-848a-1363e34c137d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:04.522880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01ec13-84af-493b-848a-1363e34c137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:04.544161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ec01ec13-84af-493b-848a-1363e34c137d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:04.545516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ec01ec13-84af-493b-848a-1363e34c137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:04.625676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 989/1975] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:04.635755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:04.637238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:04.701644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a34828-81b5-4b88-8698-bff3ad8236fb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:04.706095 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25a7cf-13c6-4eb6-84b8-3670d1fb89cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:04.706795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25a7cf-13c6-4eb6-84b8-3670d1fb89cd service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:04.707081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25a7cf-13c6-4eb6-84b8-3670d1fb89cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:04.707357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25a7cf-13c6-4eb6-84b8-3670d1fb89cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:04.717845 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25a7cf-13c6-4eb6-84b8-3670d1fb89cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:04.719281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25a7cf-13c6-4eb6-84b8-3670d1fb89cd service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:04.725066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:04.736244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 987/1976] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:05.059100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01ec13-84af-493b-848a-1363e34c137d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.060770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ec01ec13-84af-493b-848a-1363e34c137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:05.072816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01ec13-84af-493b-848a-1363e34c137d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.082302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.082980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.083253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.083547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] PATH_INFO: `/v3/projects/47bc0ace46a64934810bbc12e6c470d7` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.085446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bc0ace46a64934810bbc12e6c470d7)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.087830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.138720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bc0ace46a64934810bbc12e6c470d7', 'request_id': 'req-0ce71f0d-eed9-4e42-98d8-c8559396c280'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:05.145118 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bc0ace46a64934810bbc12e6c470d7', 'request_id': 'req-0ce71f0d-eed9-4e42-98d8-c8559396c280'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:05.151317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bc0ace46a64934810bbc12e6c470d7', 'request_id': 'req-0ce71f0d-eed9-4e42-98d8-c8559396c280'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:05.164938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] The token cache is being invalidate because project 47bc0ace46a64934810bbc12e6c470d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:05.165230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:05.165863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ce71f0d-eed9-4e42-98d8-c8559396c280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:05.167371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 990/1977] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:05 2022] DELETE /identity/v3/projects/47bc0ace46a64934810bbc12e6c470d7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:05.172318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03b53e-d058-489f-b214-73d2cd8e1974 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.172606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03b53e-d058-489f-b214-73d2cd8e1974 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.172880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03b53e-d058-489f-b214-73d2cd8e1974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.194212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2b03b53e-d058-489f-b214-73d2cd8e1974 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:05.195473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2b03b53e-d058-489f-b214-73d2cd8e1974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:05.256921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 988/1978] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:05.265107 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7b5f-be98-4235-b477-ab478fe92c49 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.265671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7b5f-be98-4235-b477-ab478fe92c49 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.266466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7b5f-be98-4235-b477-ab478fe92c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.278643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:05.298786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-2cae7b5f-be98-4235-b477-ab478fe92c49 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:05.300010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2cae7b5f-be98-4235-b477-ab478fe92c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:05.308797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 991/1979] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:05.312844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03b53e-d058-489f-b214-73d2cd8e1974 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.322614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03b53e-d058-489f-b214-73d2cd8e1974 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.327470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.328070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.328391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.328666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.329322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:05.329858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.331216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.348136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 989/1980] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:29:05.349620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae7b5f-be98-4235-b477-ab478fe92c49 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.359518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae7b5f-be98-4235-b477-ab478fe92c49 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.364671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85b16c-e7de-4236-abf7-f03f8e525e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.365324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85b16c-e7de-4236-abf7-f03f8e525e96 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.365605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85b16c-e7de-4236-abf7-f03f8e525e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.365878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85b16c-e7de-4236-abf7-f03f8e525e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.366508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85b16c-e7de-4236-abf7-f03f8e525e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.368432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85b16c-e7de-4236-abf7-f03f8e525e96 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.400144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 992/1981] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:05 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:05.404454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.415623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66404565-0948-4620-8b4f-41dfc13c5d8f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.502192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ab381-a570-4325-a5ef-e320d38597cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.502858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab381-a570-4325-a5ef-e320d38597cd admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.503143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab381-a570-4325-a5ef-e320d38597cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.503472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab381-a570-4325-a5ef-e320d38597cd admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.504243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ab381-a570-4325-a5ef-e320d38597cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.507146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ab381-a570-4325-a5ef-e320d38597cd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.532743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 990/1982] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:05 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:05.533976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85b16c-e7de-4236-abf7-f03f8e525e96 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.546404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85b16c-e7de-4236-abf7-f03f8e525e96 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.551858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50befa-3c44-4bae-bae8-68b0e7db82ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.552643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50befa-3c44-4bae-bae8-68b0e7db82ab admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.553885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50befa-3c44-4bae-bae8-68b0e7db82ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.554547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50befa-3c44-4bae-bae8-68b0e7db82ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.554818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50befa-3c44-4bae-bae8-68b0e7db82ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.556697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50befa-3c44-4bae-bae8-68b0e7db82ab admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.568271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 993/1983] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:05.573557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ab381-a570-4325-a5ef-e320d38597cd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.595124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ab381-a570-4325-a5ef-e320d38597cd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.603719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8559d0-86fa-4235-af5c-3b0abdacccdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.604670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8559d0-86fa-4235-af5c-3b0abdacccdf admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.605104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8559d0-86fa-4235-af5c-3b0abdacccdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.605537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8559d0-86fa-4235-af5c-3b0abdacccdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.606198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8559d0-86fa-4235-af5c-3b0abdacccdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.608083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8559d0-86fa-4235-af5c-3b0abdacccdf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.624421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 991/1984] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:05.632668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50befa-3c44-4bae-bae8-68b0e7db82ab admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.644243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50befa-3c44-4bae-bae8-68b0e7db82ab admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.650242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.650983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.651272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.651609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] PATH_INFO: `/v3/projects/26ff1514f2c240f0808f8aad188ac97f/users/7d3bafbdaf274f33aef3c7d066eda6c8/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.670990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ff1514f2c240f0808f8aad188ac97f, user_id=7d3bafbdaf274f33aef3c7d066eda6c8, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.674603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.699060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 994/1985] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:05 2022] PUT /identity/v3/projects/26ff1514f2c240f0808f8aad188ac97f/users/7d3bafbdaf274f33aef3c7d066eda6c8/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:05.704853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e5c8f-48c5-415e-9705-3299c2aea2ab None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.705323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e5c8f-48c5-415e-9705-3299c2aea2ab None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.705630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e5c8f-48c5-415e-9705-3299c2aea2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.738742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1a9e5c8f-48c5-415e-9705-3299c2aea2ab None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:05.741285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1a9e5c8f-48c5-415e-9705-3299c2aea2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:05.860139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 992/1986] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:05.861277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.866824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:05.874173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e5c8f-48c5-415e-9705-3299c2aea2ab None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:05.875761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1a9e5c8f-48c5-415e-9705-3299c2aea2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:05.881728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437381-efed-4cc2-9e22-849dce37f848 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.887836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.888256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:05.889249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.889663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.890040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] PATH_INFO: `/v3/users/17ea91493e324fb99cddaa0f121ad36c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.891616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17ea91493e324fb99cddaa0f121ad36c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.894172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.957910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e5c8f-48c5-415e-9705-3299c2aea2ab None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:05.963766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998121d-0b90-43fd-8b23-e5faf313ebcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:05.964410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998121d-0b90-43fd-8b23-e5faf313ebcb service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:05.964705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998121d-0b90-43fd-8b23-e5faf313ebcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:05.964985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998121d-0b90-43fd-8b23-e5faf313ebcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:05.969063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17ea91493e324fb99cddaa0f121ad36c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:05.980647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998121d-0b90-43fd-8b23-e5faf313ebcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:05.982074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998121d-0b90-43fd-8b23-e5faf313ebcb service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:05.986474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17ea91493e324fb99cddaa0f121ad36c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:05.987233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17ea91493e324fb99cddaa0f121ad36c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:05.993609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 993/1987] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:05.997890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17ea91493e324fb99cddaa0f121ad36c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:06.006384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17ea91493e324fb99cddaa0f121ad36c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:06.016456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 995/1988] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:05 2022] DELETE /identity/v3/users/17ea91493e324fb99cddaa0f121ad36c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:06.023135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94036-0518-4344-b7cb-bcb31309edfe None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:06.023135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94036-0518-4344-b7cb-bcb31309edfe None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:06.023317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94036-0518-4344-b7cb-bcb31309edfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:06.065975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-44a94036-0518-4344-b7cb-bcb31309edfe None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:06.067354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-44a94036-0518-4344-b7cb-bcb31309edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:06.135279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 994/1989] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:06.145823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:06.147905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:06.212517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7ac63-9c6d-43c9-ac67-524cb4534176 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:06.216190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911f216-926d-43f9-b688-a8e272e85cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:06.216798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911f216-926d-43f9-b688-a8e272e85cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:06.217073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911f216-926d-43f9-b688-a8e272e85cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:06.217337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911f216-926d-43f9-b688-a8e272e85cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:06.226708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911f216-926d-43f9-b688-a8e272e85cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:06.228140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911f216-926d-43f9-b688-a8e272e85cf6 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:06.237902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 996/1990] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:06.656131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94036-0518-4344-b7cb-bcb31309edfe None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:06.658015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-44a94036-0518-4344-b7cb-bcb31309edfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:06.682428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94036-0518-4344-b7cb-bcb31309edfe None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:06.689394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:06.690557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:06.691077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:06.691541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] PATH_INFO: `/v3/projects/9ff68166c3af41e2b37f3b761a1ca017` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:06.694513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff68166c3af41e2b37f3b761a1ca017)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:06.697296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:06.795912 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff68166c3af41e2b37f3b761a1ca017', 'request_id': 'req-b3080d71-7c29-413f-ba73-b3452940c8dc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:06.807973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff68166c3af41e2b37f3b761a1ca017', 'request_id': 'req-b3080d71-7c29-413f-ba73-b3452940c8dc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:06.816079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff68166c3af41e2b37f3b761a1ca017', 'request_id': 'req-b3080d71-7c29-413f-ba73-b3452940c8dc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:06.845323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] The token cache is being invalidate because project 9ff68166c3af41e2b37f3b761a1ca017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:06.845687 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:06.847450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b3080d71-7c29-413f-ba73-b3452940c8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:06.850238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 995/1991] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:06 2022] DELETE /identity/v3/projects/9ff68166c3af41e2b37f3b761a1ca017 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:06.854251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911f216-926d-43f9-b688-a8e272e85cf6 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:06.869483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911f216-926d-43f9-b688-a8e272e85cf6 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:06.879557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78166620-1c53-4fd8-9d85-84a568656112 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:06.880296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78166620-1c53-4fd8-9d85-84a568656112 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:06.881690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78166620-1c53-4fd8-9d85-84a568656112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:06.907260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:06.966231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-78166620-1c53-4fd8-9d85-84a568656112 None None] MFA Rules not processed for user `7d3bafbdaf274f33aef3c7d066eda6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:06.967735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-78166620-1c53-4fd8-9d85-84a568656112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:06.998196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:06.999226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:07.000211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:07.000485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] PATH_INFO: `/v3/users/7e10e392f53b4cd597ea8c5c433ccd8c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:07.002403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e10e392f53b4cd597ea8c5c433ccd8c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:07.004412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:07.005717 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:07.086166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 996/1992] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:07.093600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78166620-1c53-4fd8-9d85-84a568656112 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:07.104329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-78166620-1c53-4fd8-9d85-84a568656112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:07.145796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e10e392f53b4cd597ea8c5c433ccd8c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.176878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e10e392f53b4cd597ea8c5c433ccd8c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.178642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e10e392f53b4cd597ea8c5c433ccd8c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.216054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e10e392f53b4cd597ea8c5c433ccd8c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.226465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e10e392f53b4cd597ea8c5c433ccd8c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.228510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78166620-1c53-4fd8-9d85-84a568656112 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:07.233412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:07.233761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 997/1993] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:06 2022] DELETE /identity/v3/users/7e10e392f53b4cd597ea8c5c433ccd8c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:07.234422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:07.234859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:07.235128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:07.238405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:07.252261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:07.254039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:07.264947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 997/1994] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:29:07.616687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:07.626667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:07.735137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82afafc-a1d5-41b2-ab92-c7190055e147 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:07.739731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:07.740469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:07.740894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:07.741167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] PATH_INFO: `/v3/projects/56ff09483ec8431b8f0e1902a08b33d8` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:07.742285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ff09483ec8431b8f0e1902a08b33d8)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:07.744290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:07.782597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:07.792699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ff09483ec8431b8f0e1902a08b33d8', 'request_id': 'req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.798838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ff09483ec8431b8f0e1902a08b33d8', 'request_id': 'req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.804484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ff09483ec8431b8f0e1902a08b33d8', 'request_id': 'req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.816986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] The token cache is being invalidate because project 56ff09483ec8431b8f0e1902a08b33d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:07.817282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.817901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.819919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 998/1995] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:07 2022] DELETE /identity/v3/projects/56ff09483ec8431b8f0e1902a08b33d8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:07.823927 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:07.834626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92088b-5c70-4fc9-afc3-c379b8b31166 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:07.890385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:07.890925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:07.891203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:07.891476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] PATH_INFO: `/v3/users/ad3ff77757ad49f68271c3ad918157b0` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:07.892676 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3ff77757ad49f68271c3ad918157b0)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:07.894593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:07.942743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3ff77757ad49f68271c3ad918157b0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.951449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3ff77757ad49f68271c3ad918157b0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.952261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3ff77757ad49f68271c3ad918157b0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.963385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:07.966588 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3ff77757ad49f68271c3ad918157b0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.967533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42d921c3-c027-4113-9b1f-724c756d21d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3ff77757ad49f68271c3ad918157b0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:07.983212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 998/1996] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:07 2022] DELETE /identity/v3/users/ad3ff77757ad49f68271c3ad918157b0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:08.291166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:08.292723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:08.305963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee26af-1ee7-4dc5-9f15-bfe9f0e8f877 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:08.375406 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:08.376252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.376529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.376788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] PATH_INFO: `/v3/projects/c1769ef0ec554ac2aabb536ac158be96` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.377938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1769ef0ec554ac2aabb536ac158be96)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:08.379828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:08.438850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1769ef0ec554ac2aabb536ac158be96', 'request_id': 'req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:08.447113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1769ef0ec554ac2aabb536ac158be96', 'request_id': 'req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:08.453925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1769ef0ec554ac2aabb536ac158be96', 'request_id': 'req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:08.467513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] The token cache is being invalidate because project c1769ef0ec554ac2aabb536ac158be96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:08.467940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:08.468683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6eae385e-c9c5-4dae-bc8a-adece15a5f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:08.470388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 999/1997] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:08 2022] DELETE /identity/v3/projects/c1769ef0ec554ac2aabb536ac158be96 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:08.476714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddc407-ca69-435d-8449-d41787227b65 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.477169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddc407-ca69-435d-8449-d41787227b65 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.477507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcddc407-ca69-435d-8449-d41787227b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.497918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-dcddc407-ca69-435d-8449-d41787227b65 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:08.499256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dcddc407-ca69-435d-8449-d41787227b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:08.559190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 999/1998] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:08.567417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c8f18-f44a-4d60-b8e0-fb6794752da8 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.567697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c8f18-f44a-4d60-b8e0-fb6794752da8 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.568000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c8f18-f44a-4d60-b8e0-fb6794752da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.593823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-bc3c8f18-f44a-4d60-b8e0-fb6794752da8 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:08.595329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-bc3c8f18-f44a-4d60-b8e0-fb6794752da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:08.605331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1000/1999] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:08.609633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddc407-ca69-435d-8449-d41787227b65 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:08.620385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcddc407-ca69-435d-8449-d41787227b65 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:08.625040 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478cba4b-5294-456a-89ad-cb233484fcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:08.625648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cba4b-5294-456a-89ad-cb233484fcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.625928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cba4b-5294-456a-89ad-cb233484fcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.626201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478cba4b-5294-456a-89ad-cb233484fcc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.626846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cba4b-5294-456a-89ad-cb233484fcc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:08.627175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cba4b-5294-456a-89ad-cb233484fcc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:08.628854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478cba4b-5294-456a-89ad-cb233484fcc6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:08.644138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1000/2000] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:29:08.645289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c8f18-f44a-4d60-b8e0-fb6794752da8 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:08.656307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c8f18-f44a-4d60-b8e0-fb6794752da8 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:08.664296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35e70b-08a3-4377-ac38-5c455491cc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:08.665244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35e70b-08a3-4377-ac38-5c455491cc53 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.665713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35e70b-08a3-4377-ac38-5c455491cc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.666188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35e70b-08a3-4377-ac38-5c455491cc53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.667016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35e70b-08a3-4377-ac38-5c455491cc53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:08.669994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35e70b-08a3-4377-ac38-5c455491cc53 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:08.697631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c60a1-ff62-475b-9ddd-7a37e691600d None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.698101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c60a1-ff62-475b-9ddd-7a37e691600d None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.698424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c60a1-ff62-475b-9ddd-7a37e691600d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.703696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1001/2001] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:08 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:08.709094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35e70b-08a3-4377-ac38-5c455491cc53 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:08.722021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-215c60a1-ff62-475b-9ddd-7a37e691600d None None] MFA Rules not processed for user `7d3bafbdaf274f33aef3c7d066eda6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:08.723354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-215c60a1-ff62-475b-9ddd-7a37e691600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:08.750341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:08.799777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1001/2002] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:08.808273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35e70b-08a3-4377-ac38-5c455491cc53 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:08.818737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66142f85-c822-49ae-995e-057ae93616c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:08.819927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66142f85-c822-49ae-995e-057ae93616c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.820427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66142f85-c822-49ae-995e-057ae93616c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.820804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66142f85-c822-49ae-995e-057ae93616c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.821532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66142f85-c822-49ae-995e-057ae93616c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:08.823666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66142f85-c822-49ae-995e-057ae93616c2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:08.851196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1002/2003] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:08 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:08.855323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c60a1-ff62-475b-9ddd-7a37e691600d None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:08.866325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c60a1-ff62-475b-9ddd-7a37e691600d None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:08.872066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31f9ec-383a-4fc4-9fa5-c785120978cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:08.872864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31f9ec-383a-4fc4-9fa5-c785120978cd admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.873300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31f9ec-383a-4fc4-9fa5-c785120978cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.873723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31f9ec-383a-4fc4-9fa5-c785120978cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.874509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31f9ec-383a-4fc4-9fa5-c785120978cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:08.876616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31f9ec-383a-4fc4-9fa5-c785120978cd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:08.887795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1002/2004] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:29:08.892047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66142f85-c822-49ae-995e-057ae93616c2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:08.905357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66142f85-c822-49ae-995e-057ae93616c2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:08.911236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a5d2a-57f6-410d-9dcc-f88b0e17e535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:08.911984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a5d2a-57f6-410d-9dcc-f88b0e17e535 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.912994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a5d2a-57f6-410d-9dcc-f88b0e17e535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.913211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a5d2a-57f6-410d-9dcc-f88b0e17e535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.913928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a5d2a-57f6-410d-9dcc-f88b0e17e535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:08.915672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2a5d2a-57f6-410d-9dcc-f88b0e17e535 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:08.928150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1003/2005] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:08.931681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31f9ec-383a-4fc4-9fa5-c785120978cd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:08.941999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31f9ec-383a-4fc4-9fa5-c785120978cd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:08.948647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:08.949296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.950078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.950078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] PATH_INFO: `/v3/projects/5ceb65749ae9426795e7c058a7b393e6/users/920fb51213db4f14adc696bf269aabed/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:08.965622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ceb65749ae9426795e7c058a7b393e6, user_id=920fb51213db4f14adc696bf269aabed, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:08.968301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:08.982207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1003/2006] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:08 2022] PUT /identity/v3/projects/5ceb65749ae9426795e7c058a7b393e6/users/920fb51213db4f14adc696bf269aabed/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:08.994318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556858f0-ce1e-4043-b512-a2ac9bf08475 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:08.994641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556858f0-ce1e-4043-b512-a2ac9bf08475 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:08.994920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556858f0-ce1e-4043-b512-a2ac9bf08475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:09.023982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-556858f0-ce1e-4043-b512-a2ac9bf08475 None None] MFA Rules not processed for user `920fb51213db4f14adc696bf269aabed`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:09.025225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-556858f0-ce1e-4043-b512-a2ac9bf08475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:09.107952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1004/2007] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:09.119652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:09.121417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:09.195672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:09.218462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032203a-e719-4d25-8eb0-7259872e9e78 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:09.223582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac756c2-2780-401a-9896-22cab9c7640f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:09.224423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac756c2-2780-401a-9896-22cab9c7640f service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:09.225458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac756c2-2780-401a-9896-22cab9c7640f service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:09.225458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac756c2-2780-401a-9896-22cab9c7640f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:09.246005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac756c2-2780-401a-9896-22cab9c7640f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:09.247439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac756c2-2780-401a-9896-22cab9c7640f service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:09.257878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1004/2008] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:19.105131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=54,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:19.252853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:20.369856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556858f0-ce1e-4043-b512-a2ac9bf08475 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:20.454928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556858f0-ce1e-4043-b512-a2ac9bf08475 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:20.460567 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:20.461337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:20.461864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:20.462211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] PATH_INFO: `/v3/users/920fb51213db4f14adc696bf269aabed` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:20.463465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] RBAC: Authorizing `identity:delete_user(user_id=920fb51213db4f14adc696bf269aabed)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:20.465585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:20.524031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920fb51213db4f14adc696bf269aabed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:20.549746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920fb51213db4f14adc696bf269aabed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:20.550549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920fb51213db4f14adc696bf269aabed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:20.560152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920fb51213db4f14adc696bf269aabed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:20.568986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920fb51213db4f14adc696bf269aabed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:20.575387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1005/2009] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:20 2022] DELETE /identity/v3/users/920fb51213db4f14adc696bf269aabed => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:20.580012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a167101-c6b2-439e-a54a-a7d15d751c6f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:20.580831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a167101-c6b2-439e-a54a-a7d15d751c6f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:20.581103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a167101-c6b2-439e-a54a-a7d15d751c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:20.601765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9a167101-c6b2-439e-a54a-a7d15d751c6f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:20.602976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9a167101-c6b2-439e-a54a-a7d15d751c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:20.661199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1005/2010] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:20.671518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:20.673021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:20.731461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2fca7-1899-404e-ad00-4f5e831eb32f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:20.735111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:20.735710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:20.735985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:20.736287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:20.745588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:20.746996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:20.756591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1006/2011] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:21.295363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a167101-c6b2-439e-a54a-a7d15d751c6f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:21.296826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9a167101-c6b2-439e-a54a-a7d15d751c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:21.309663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a167101-c6b2-439e-a54a-a7d15d751c6f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:21.314525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:21.315207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:21.315509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:21.315776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] PATH_INFO: `/v3/projects/5ceb65749ae9426795e7c058a7b393e6` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:21.320244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ceb65749ae9426795e7c058a7b393e6)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:21.322913 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:21.370870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ceb65749ae9426795e7c058a7b393e6', 'request_id': 'req-8527da61-0da3-4e66-9b7f-f9450f529599'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:21.379536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ceb65749ae9426795e7c058a7b393e6', 'request_id': 'req-8527da61-0da3-4e66-9b7f-f9450f529599'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:21.387204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ceb65749ae9426795e7c058a7b393e6', 'request_id': 'req-8527da61-0da3-4e66-9b7f-f9450f529599'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:21.398103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] The token cache is being invalidate because project 5ceb65749ae9426795e7c058a7b393e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:21.398393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:21.399069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8527da61-0da3-4e66-9b7f-f9450f529599 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:21.400638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1006/2012] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:21 2022] DELETE /identity/v3/projects/5ceb65749ae9426795e7c058a7b393e6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:24.378930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:24.381028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:24.403581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120cee8b-6341-496d-ac37-73d64056ad81 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:24.428422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:24.480198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:24.480198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:24.480526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:24.480630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] PATH_INFO: `/v3/users/742b13f45eb54d90a82b409111ff64f5` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:24.481705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] RBAC: Authorizing `identity:delete_user(user_id=742b13f45eb54d90a82b409111ff64f5)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:24.483613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:24.548123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742b13f45eb54d90a82b409111ff64f5'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:24.575611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742b13f45eb54d90a82b409111ff64f5'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:24.576873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742b13f45eb54d90a82b409111ff64f5'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:24.586929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742b13f45eb54d90a82b409111ff64f5'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:24.605460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742b13f45eb54d90a82b409111ff64f5'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:24.618861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1007/2013] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:24 2022] DELETE /identity/v3/users/742b13f45eb54d90a82b409111ff64f5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:24.620857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99515-ff25-461a-b318-779118e5e539 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:24.621374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99515-ff25-461a-b318-779118e5e539 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:24.621458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99515-ff25-461a-b318-779118e5e539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:24.655752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-75f99515-ff25-461a-b318-779118e5e539 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:24.657280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-75f99515-ff25-461a-b318-779118e5e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:24.746137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1007/2014] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:24.767239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:24.767239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:24.834704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6019b-f3e0-4647-a1e8-6a781df2f27c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:24.838901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c11e3-1ec7-4c4d-869e-70e8675aae13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:24.839152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c11e3-1ec7-4c4d-869e-70e8675aae13 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:24.839376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c11e3-1ec7-4c4d-869e-70e8675aae13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:24.839675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c11e3-1ec7-4c4d-869e-70e8675aae13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:24.851263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c11e3-1ec7-4c4d-869e-70e8675aae13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:24.852739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c11e3-1ec7-4c4d-869e-70e8675aae13 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:24.862765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1008/2015] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:25.510331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99515-ff25-461a-b318-779118e5e539 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:25.511836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-75f99515-ff25-461a-b318-779118e5e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:25.534784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99515-ff25-461a-b318-779118e5e539 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:25.542006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:25.542789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:25.543071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:25.543356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] PATH_INFO: `/v3/projects/68eba302ba704d6eae134f3fe53935c4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:25.545404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68eba302ba704d6eae134f3fe53935c4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:25.548618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:25.575514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:25.613815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68eba302ba704d6eae134f3fe53935c4', 'request_id': 'req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:25.621161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68eba302ba704d6eae134f3fe53935c4', 'request_id': 'req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:25.627764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68eba302ba704d6eae134f3fe53935c4', 'request_id': 'req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:25.638175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] The token cache is being invalidate because project 68eba302ba704d6eae134f3fe53935c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:25.638465 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:25.639116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e53de17c-8a80-4b9e-b241-d48ce5afc0f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:25.640660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1008/2016] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:25 2022] DELETE /identity/v3/projects/68eba302ba704d6eae134f3fe53935c4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:25.645565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10b618-45aa-496e-ad8e-169e9c8a5975 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:25.646067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10b618-45aa-496e-ad8e-169e9c8a5975 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:25.646304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10b618-45aa-496e-ad8e-169e9c8a5975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:25.666375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-fe10b618-45aa-496e-ad8e-169e9c8a5975 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:25.667574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fe10b618-45aa-496e-ad8e-169e9c8a5975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:25.710384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:25.732660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1009/2017] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:25.740542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec13c6-79e1-409a-a73d-a3082509aac1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:25.740939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec13c6-79e1-409a-a73d-a3082509aac1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:25.741243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec13c6-79e1-409a-a73d-a3082509aac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:25.760816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-28ec13c6-79e1-409a-a73d-a3082509aac1 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:25.762025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-28ec13c6-79e1-409a-a73d-a3082509aac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:25.771164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1009/2018] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:25.775564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10b618-45aa-496e-ad8e-169e9c8a5975 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:25.787265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10b618-45aa-496e-ad8e-169e9c8a5975 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:25.795877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:25.796599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:25.796937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:25.797471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:25.798354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:25.798815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:25.800897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:25.816902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1010/2019] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:29:25.821257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec13c6-79e1-409a-a73d-a3082509aac1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:25.831507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec13c6-79e1-409a-a73d-a3082509aac1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:25.837885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f516ccd-abf4-4547-ae9d-e7ae5405cdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:25.838441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f516ccd-abf4-4547-ae9d-e7ae5405cdde admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:25.838723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f516ccd-abf4-4547-ae9d-e7ae5405cdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:25.839084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f516ccd-abf4-4547-ae9d-e7ae5405cdde admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:25.839634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f516ccd-abf4-4547-ae9d-e7ae5405cdde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:25.841575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f516ccd-abf4-4547-ae9d-e7ae5405cdde admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:25.875377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1010/2020] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:25 2022] POST /identity/v3/projects => generated 359 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:25.876641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:25.887445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d16f49-8f11-44c5-8dce-ce433da9e2a0 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:25.984905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94212607-e266-4da2-ac33-3544a4f1dbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:25.985511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212607-e266-4da2-ac33-3544a4f1dbf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:25.985843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212607-e266-4da2-ac33-3544a4f1dbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:25.986121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94212607-e266-4da2-ac33-3544a4f1dbf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:25.986760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94212607-e266-4da2-ac33-3544a4f1dbf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:25.988923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94212607-e266-4da2-ac33-3544a4f1dbf0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:26.024136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1011/2021] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:25 2022] POST /identity/v3/users => generated 280 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:26.025410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f516ccd-abf4-4547-ae9d-e7ae5405cdde admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:26.037066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f516ccd-abf4-4547-ae9d-e7ae5405cdde admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:26.048866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75103-912a-4452-b982-d5099e3143b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:26.049532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75103-912a-4452-b982-d5099e3143b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:26.049822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75103-912a-4452-b982-d5099e3143b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:26.050114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb75103-912a-4452-b982-d5099e3143b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:26.050767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb75103-912a-4452-b982-d5099e3143b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:26.052474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb75103-912a-4452-b982-d5099e3143b6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:26.064835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1011/2022] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:26.069791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94212607-e266-4da2-ac33-3544a4f1dbf0 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:26.082851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94212607-e266-4da2-ac33-3544a4f1dbf0 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:26.090983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ee07d-12d5-4bb6-a778-6885e75c8d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:26.091610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee07d-12d5-4bb6-a778-6885e75c8d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:26.092009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee07d-12d5-4bb6-a778-6885e75c8d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:26.092220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee07d-12d5-4bb6-a778-6885e75c8d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:26.092834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ee07d-12d5-4bb6-a778-6885e75c8d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:26.094431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ee07d-12d5-4bb6-a778-6885e75c8d49 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:26.114791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1012/2023] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:26.117204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75103-912a-4452-b982-d5099e3143b6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:26.129028 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb75103-912a-4452-b982-d5099e3143b6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:26.135371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:26.136149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:26.136501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:26.136855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] PATH_INFO: `/v3/projects/170bb6e253a34f12a851b1705cfbf97d/users/8061fcac9ec54b2c91af4356259c31de/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:26.153586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170bb6e253a34f12a851b1705cfbf97d, user_id=8061fcac9ec54b2c91af4356259c31de, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:26.159271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:26.176514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1012/2024] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:26 2022] PUT /identity/v3/projects/170bb6e253a34f12a851b1705cfbf97d/users/8061fcac9ec54b2c91af4356259c31de/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:26.188303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09371c-7f19-4d3e-865e-be267201f5f4 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:26.188592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09371c-7f19-4d3e-865e-be267201f5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:26.188870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09371c-7f19-4d3e-865e-be267201f5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:26.231792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3b09371c-7f19-4d3e-865e-be267201f5f4 None None] MFA Rules not processed for user `8061fcac9ec54b2c91af4356259c31de`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:26.233222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3b09371c-7f19-4d3e-865e-be267201f5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:26.328736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1013/2025] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:26.364190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:26.366619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:26.440365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7570218-82d1-4943-9f1a-998630544884 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:26.444951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cdf73-5708-41a1-af28-ad5937c1b36b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:26.445571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cdf73-5708-41a1-af28-ad5937c1b36b service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:26.446555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cdf73-5708-41a1-af28-ad5937c1b36b service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:26.446836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cdf73-5708-41a1-af28-ad5937c1b36b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:26.459424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cdf73-5708-41a1-af28-ad5937c1b36b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:26.460897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cdf73-5708-41a1-af28-ad5937c1b36b service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:26.472776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1013/2026] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:26.492605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09371c-7f19-4d3e-865e-be267201f5f4 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:26.549559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:26.575656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09371c-7f19-4d3e-865e-be267201f5f4 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:26.580685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:26.581390 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:26.581733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:26.582068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] PATH_INFO: `/v3/users/8061fcac9ec54b2c91af4356259c31de` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:26.583855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8061fcac9ec54b2c91af4356259c31de)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:26.586111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:26.646957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8061fcac9ec54b2c91af4356259c31de'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:26.662524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8061fcac9ec54b2c91af4356259c31de'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:26.663357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8061fcac9ec54b2c91af4356259c31de'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:26.672399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8061fcac9ec54b2c91af4356259c31de'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:26.684421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8061fcac9ec54b2c91af4356259c31de'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:26.695492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1014/2027] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:26 2022] DELETE /identity/v3/users/8061fcac9ec54b2c91af4356259c31de => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:26.697903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757342ca-19a5-4f25-bc0f-487e29ddc1e6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:26.698441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757342ca-19a5-4f25-bc0f-487e29ddc1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:26.698579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757342ca-19a5-4f25-bc0f-487e29ddc1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:26.718777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-757342ca-19a5-4f25-bc0f-487e29ddc1e6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:26.719973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-757342ca-19a5-4f25-bc0f-487e29ddc1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:26.763689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:26.783137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1014/2028] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:26.801290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:26.803073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:26.864538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1ece0-5646-4600-b95e-16a7152cffb7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:26.868276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e603ce-bdf3-4158-9c38-450790babacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:26.868876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e603ce-bdf3-4158-9c38-450790babacb service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:26.869147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e603ce-bdf3-4158-9c38-450790babacb service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:26.869408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e603ce-bdf3-4158-9c38-450790babacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:26.878825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e603ce-bdf3-4158-9c38-450790babacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:26.880921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e603ce-bdf3-4158-9c38-450790babacb service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:26.891047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1015/2029] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:27.329643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138e2d-9b6b-470b-a38e-a6969b27e5af None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:27.329928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138e2d-9b6b-470b-a38e-a6969b27e5af None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:27.330198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9138e2d-9b6b-470b-a38e-a6969b27e5af None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:27.331586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1015/2030] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:29:27.374717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9d2fd-a35a-4429-a3e0-44279c9873b1 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:27.375211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9d2fd-a35a-4429-a3e0-44279c9873b1 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:27.375492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9d2fd-a35a-4429-a3e0-44279c9873b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:27.396765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-2eb9d2fd-a35a-4429-a3e0-44279c9873b1 None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:27.398192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2eb9d2fd-a35a-4429-a3e0-44279c9873b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:27.456767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:27.472285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1016/2031] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:27.495462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138e2d-9b6b-470b-a38e-a6969b27e5af None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:27.503557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a9138e2d-9b6b-470b-a38e-a6969b27e5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:27.518670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9d2fd-a35a-4429-a3e0-44279c9873b1 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:27.522561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2eb9d2fd-a35a-4429-a3e0-44279c9873b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:27.545386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9138e2d-9b6b-470b-a38e-a6969b27e5af None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:27.551347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:27.552080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:27.552381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:27.552654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] PATH_INFO: `/v3/users/eca40eca0b6140a498e034e3e37a8c47` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:27.553644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9d2fd-a35a-4429-a3e0-44279c9873b1 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:27.553800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca40eca0b6140a498e034e3e37a8c47)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:27.555750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:27.568148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:27.568790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:27.569069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:27.569393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:27.579712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:27.581338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:27.595142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1017/2032] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:27.633235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca40eca0b6140a498e034e3e37a8c47'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.642741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca40eca0b6140a498e034e3e37a8c47'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.643905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca40eca0b6140a498e034e3e37a8c47'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.658361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca40eca0b6140a498e034e3e37a8c47'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.659428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca40eca0b6140a498e034e3e37a8c47'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.675364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1016/2033] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:27 2022] DELETE /identity/v3/users/eca40eca0b6140a498e034e3e37a8c47 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:27.763296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:27.766217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:27.782217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48f7fb-265f-4f7a-a313-0269a8799b42 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:27.843113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:27.843865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:27.844243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:27.844577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] PATH_INFO: `/v3/projects/170bb6e253a34f12a851b1705cfbf97d` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:27.845915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=170bb6e253a34f12a851b1705cfbf97d)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:27.847836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:27.868885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:27.870867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:27.889051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1d8ab-c268-4167-abfd-5b66ff926f38 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:27.912202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170bb6e253a34f12a851b1705cfbf97d', 'request_id': 'req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.919126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170bb6e253a34f12a851b1705cfbf97d', 'request_id': 'req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.924981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170bb6e253a34f12a851b1705cfbf97d', 'request_id': 'req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.937935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] The token cache is being invalidate because project 170bb6e253a34f12a851b1705cfbf97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:27.938274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.939208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0e5bb57f-7c32-4a63-a740-54e21bb4ffd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:27.941057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1018/2034] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:27 2022] DELETE /identity/v3/projects/170bb6e253a34f12a851b1705cfbf97d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:27.947107 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264c8df-fb29-44b6-889a-2904ff05d2a7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:27.947415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264c8df-fb29-44b6-889a-2904ff05d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:27.947710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264c8df-fb29-44b6-889a-2904ff05d2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:27.952372 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:27.953014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:27.953524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:27.953807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] PATH_INFO: `/v3/projects/77b79bd8e6b84cd3a082d80b4aa3a546` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:27.954937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b79bd8e6b84cd3a082d80b4aa3a546)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:27.957031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:27.969258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-a264c8df-fb29-44b6-889a-2904ff05d2a7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:27.970460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a264c8df-fb29-44b6-889a-2904ff05d2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:27.978963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1019/2035] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:27.986716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aae3c-2818-4469-a584-ed4eb8732ca0 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:27.987408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aae3c-2818-4469-a584-ed4eb8732ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:27.987685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583aae3c-2818-4469-a584-ed4eb8732ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.012773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-583aae3c-2818-4469-a584-ed4eb8732ca0 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:28.014285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-583aae3c-2818-4469-a584-ed4eb8732ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:28.015343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b79bd8e6b84cd3a082d80b4aa3a546', 'request_id': 'req-cc886a41-6723-4011-8045-4121d7de322d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:28.025645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b79bd8e6b84cd3a082d80b4aa3a546', 'request_id': 'req-cc886a41-6723-4011-8045-4121d7de322d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:28.032986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b79bd8e6b84cd3a082d80b4aa3a546', 'request_id': 'req-cc886a41-6723-4011-8045-4121d7de322d'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:28.034564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:28.061965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] The token cache is being invalidate because project 77b79bd8e6b84cd3a082d80b4aa3a546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:28.062490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:28.063543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cc886a41-6723-4011-8045-4121d7de322d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:28.065189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1017/2036] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:27 2022] DELETE /identity/v3/projects/77b79bd8e6b84cd3a082d80b4aa3a546 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:28.078613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6223-f2d2-407d-9745-74ca06bc4128 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.079241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6223-f2d2-407d-9745-74ca06bc4128 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.080115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6223-f2d2-407d-9745-74ca06bc4128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.101609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1020/2037] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:28.105462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aae3c-2818-4469-a584-ed4eb8732ca0 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.105620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fb1f6223-f2d2-407d-9745-74ca06bc4128 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:28.106919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fb1f6223-f2d2-407d-9745-74ca06bc4128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:28.116194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1018/2038] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:28.120759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583aae3c-2818-4469-a584-ed4eb8732ca0 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.125463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abe4f6-8a2b-4124-932a-7daf664992e0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.125750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abe4f6-8a2b-4124-932a-7daf664992e0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.125927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.126043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abe4f6-8a2b-4124-932a-7daf664992e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.126696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.127038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.127371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.128143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:28.128508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.130859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.133723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:28.143920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1021/2039] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:29:28.149156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.149961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-96abe4f6-8a2b-4124-932a-7daf664992e0 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:28.151750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-96abe4f6-8a2b-4124-932a-7daf664992e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:28.159837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1019/2040] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:28.160949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420cb6b3-d813-4274-a5ad-e379426fcba9 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.164268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abe4f6-8a2b-4124-932a-7daf664992e0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.166195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378cc14-c4a1-4b4e-8b07-4ce87ddc21f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.166797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378cc14-c4a1-4b4e-8b07-4ce87ddc21f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.167084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378cc14-c4a1-4b4e-8b07-4ce87ddc21f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.167334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378cc14-c4a1-4b4e-8b07-4ce87ddc21f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.167978 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378cc14-c4a1-4b4e-8b07-4ce87ddc21f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.169904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378cc14-c4a1-4b4e-8b07-4ce87ddc21f4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.173853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abe4f6-8a2b-4124-932a-7daf664992e0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.179239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.179916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.180296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.180573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.181528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:28.181840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.183433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.195918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1020/2041] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:29:28.198863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1022/2042] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:28.200162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.202653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378cc14-c4a1-4b4e-8b07-4ce87ddc21f4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.210434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04565e-cdb4-46cf-91a1-a25ab7355c79 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.268034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378cc14-c4a1-4b4e-8b07-4ce87ddc21f4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.271837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42272ffb-5b2c-4cd0-a867-cf8378e5e76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.272581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42272ffb-5b2c-4cd0-a867-cf8378e5e76f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.272906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490e60b-0d16-4bf7-9170-bc4038e53fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.273052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42272ffb-5b2c-4cd0-a867-cf8378e5e76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.273342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42272ffb-5b2c-4cd0-a867-cf8378e5e76f admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.273495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490e60b-0d16-4bf7-9170-bc4038e53fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.273770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490e60b-0d16-4bf7-9170-bc4038e53fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.274035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490e60b-0d16-4bf7-9170-bc4038e53fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.274137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42272ffb-5b2c-4cd0-a867-cf8378e5e76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.274647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490e60b-0d16-4bf7-9170-bc4038e53fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.275987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42272ffb-5b2c-4cd0-a867-cf8378e5e76f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.276754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490e60b-0d16-4bf7-9170-bc4038e53fd4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.299689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1023/2043] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:28.304080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490e60b-0d16-4bf7-9170-bc4038e53fd4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.304506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1021/2044] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/projects => generated 355 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:28.308308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42272ffb-5b2c-4cd0-a867-cf8378e5e76f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.318530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490e60b-0d16-4bf7-9170-bc4038e53fd4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.380595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600042d-c37a-4f21-ab17-14524b897822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.381217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600042d-c37a-4f21-ab17-14524b897822 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.381494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600042d-c37a-4f21-ab17-14524b897822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.382215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600042d-c37a-4f21-ab17-14524b897822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.382879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600042d-c37a-4f21-ab17-14524b897822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.384490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600042d-c37a-4f21-ab17-14524b897822 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.389077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42272ffb-5b2c-4cd0-a867-cf8378e5e76f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.393623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e5ce5-41f7-4ef9-89f3-3352b012ff7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.394223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e5ce5-41f7-4ef9-89f3-3352b012ff7b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.394499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e5ce5-41f7-4ef9-89f3-3352b012ff7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.394683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1024/2045] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:29:28.395899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04d95-5112-425b-8368-846bf0f7bc96 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.396193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04d95-5112-425b-8368-846bf0f7bc96 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.396445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e5ce5-41f7-4ef9-89f3-3352b012ff7b admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.396541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04d95-5112-425b-8368-846bf0f7bc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.397232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e5ce5-41f7-4ef9-89f3-3352b012ff7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.399581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e5ce5-41f7-4ef9-89f3-3352b012ff7b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.415587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-8ff04d95-5112-425b-8368-846bf0f7bc96 None None] MFA Rules not processed for user `7d3bafbdaf274f33aef3c7d066eda6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:28.417101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8ff04d95-5112-425b-8368-846bf0f7bc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:28.423308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1022/2046] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/users => generated 278 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:28.424339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e5ce5-41f7-4ef9-89f3-3352b012ff7b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.434686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e5ce5-41f7-4ef9-89f3-3352b012ff7b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.439298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53280272-8e51-49cf-8c1d-d4147e6d452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.439884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53280272-8e51-49cf-8c1d-d4147e6d452e admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.440185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53280272-8e51-49cf-8c1d-d4147e6d452e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.440453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53280272-8e51-49cf-8c1d-d4147e6d452e admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.441082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53280272-8e51-49cf-8c1d-d4147e6d452e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.442700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53280272-8e51-49cf-8c1d-d4147e6d452e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.452580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1023/2047] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:28.453425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53280272-8e51-49cf-8c1d-d4147e6d452e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.462938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53280272-8e51-49cf-8c1d-d4147e6d452e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.467563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06771d73-ef90-4138-8578-8d2848351c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.468159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771d73-ef90-4138-8578-8d2848351c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.468438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771d73-ef90-4138-8578-8d2848351c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.468701 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771d73-ef90-4138-8578-8d2848351c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.469318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06771d73-ef90-4138-8578-8d2848351c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.470847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06771d73-ef90-4138-8578-8d2848351c65 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.479168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1025/2048] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:28.479926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff04d95-5112-425b-8368-846bf0f7bc96 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.486204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:28.486615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1024/2049] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:28.491037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06771d73-ef90-4138-8578-8d2848351c65 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.494758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff04d95-5112-425b-8368-846bf0f7bc96 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.500327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e0f42-9316-4813-bd31-2143fe626a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.500992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0f42-9316-4813-bd31-2143fe626a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.501275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0f42-9316-4813-bd31-2143fe626a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.501619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e0f42-9316-4813-bd31-2143fe626a21 admin admin] PATH_INFO: `/v3/projects/ea2bfe1166f74bb290425cd93ea4e333/users/6e3802355e7c4a099d9d8659b431f4ae/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.502777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06771d73-ef90-4138-8578-8d2848351c65 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.509232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629ed13-2839-46a5-8722-d69ed61529c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.509677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ed13-2839-46a5-8722-d69ed61529c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.509956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ed13-2839-46a5-8722-d69ed61529c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.510713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629ed13-2839-46a5-8722-d69ed61529c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.511433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629ed13-2839-46a5-8722-d69ed61529c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.513225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629ed13-2839-46a5-8722-d69ed61529c3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.518020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e0f42-9316-4813-bd31-2143fe626a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea2bfe1166f74bb290425cd93ea4e333, user_id=6e3802355e7c4a099d9d8659b431f4ae, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.521255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e0f42-9316-4813-bd31-2143fe626a21 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.524582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1025/2050] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:28.528916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629ed13-2839-46a5-8722-d69ed61529c3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.537161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:28.538815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1026/2051] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:28 2022] PUT /identity/v3/projects/ea2bfe1166f74bb290425cd93ea4e333/users/6e3802355e7c4a099d9d8659b431f4ae/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:28.542129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629ed13-2839-46a5-8722-d69ed61529c3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.549804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df6ba7-a660-4575-8fdd-d6e52015bd48 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.550287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df6ba7-a660-4575-8fdd-d6e52015bd48 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.550556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df6ba7-a660-4575-8fdd-d6e52015bd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.584391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-27df6ba7-a660-4575-8fdd-d6e52015bd48 None None] MFA Rules not processed for user `6e3802355e7c4a099d9d8659b431f4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:28.585744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-27df6ba7-a660-4575-8fdd-d6e52015bd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:28.649471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5347921-e21a-4a13-9266-f165e6fba63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.651236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5347921-e21a-4a13-9266-f165e6fba63d admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.651660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5347921-e21a-4a13-9266-f165e6fba63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.652240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5347921-e21a-4a13-9266-f165e6fba63d admin admin] PATH_INFO: `/v3/projects/9518076a0c7d4c15919f7c1ffda446be/users/8cc4f4a8284d4112a675bd8ef608e9e8/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.669970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1027/2052] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:28.674517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5347921-e21a-4a13-9266-f165e6fba63d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9518076a0c7d4c15919f7c1ffda446be, user_id=8cc4f4a8284d4112a675bd8ef608e9e8, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.682583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df6ba7-a660-4575-8fdd-d6e52015bd48 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.684251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5347921-e21a-4a13-9266-f165e6fba63d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.684402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-27df6ba7-a660-4575-8fdd-d6e52015bd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:28.704490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1026/2053] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:28 2022] PUT /identity/v3/projects/9518076a0c7d4c15919f7c1ffda446be/users/8cc4f4a8284d4112a675bd8ef608e9e8/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:28.716450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233c09-0da4-4e39-9ceb-71e159cc7337 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.716665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233c09-0da4-4e39-9ceb-71e159cc7337 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.717084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f233c09-0da4-4e39-9ceb-71e159cc7337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.759840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8f233c09-0da4-4e39-9ceb-71e159cc7337 None None] MFA Rules not processed for user `8cc4f4a8284d4112a675bd8ef608e9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:28.759840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8f233c09-0da4-4e39-9ceb-71e159cc7337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:28.776987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df6ba7-a660-4575-8fdd-d6e52015bd48 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.839403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bd93b-e5c4-4a91-bb8c-a88f5f862adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.839970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bd93b-e5c4-4a91-bb8c-a88f5f862adf service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.840293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bd93b-e5c4-4a91-bb8c-a88f5f862adf service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.840565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bd93b-e5c4-4a91-bb8c-a88f5f862adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.845319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1027/2054] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:28.846324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f233c09-0da4-4e39-9ceb-71e159cc7337 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.896796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:28.914432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f233c09-0da4-4e39-9ceb-71e159cc7337 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:28.917733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bd93b-e5c4-4a91-bb8c-a88f5f862adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.919580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bd93b-e5c4-4a91-bb8c-a88f5f862adf service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.920643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e63008-fc22-489b-aec0-1c23ccdda4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:28.921259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e63008-fc22-489b-aec0-1c23ccdda4ef admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:28.921566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e63008-fc22-489b-aec0-1c23ccdda4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:28.921842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e63008-fc22-489b-aec0-1c23ccdda4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:28.922482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e63008-fc22-489b-aec0-1c23ccdda4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:28.924411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e63008-fc22-489b-aec0-1c23ccdda4ef admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:28.930627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1028/2055] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:29:28.960452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1028/2056] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/projects => generated 357 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:28.961488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bd93b-e5c4-4a91-bb8c-a88f5f862adf service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.962097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e63008-fc22-489b-aec0-1c23ccdda4ef admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:28.966884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:28.975670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bd93b-e5c4-4a91-bb8c-a88f5f862adf service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.037469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a8324-4b0c-4d6a-b5d9-608c69530414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.038087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a8324-4b0c-4d6a-b5d9-608c69530414 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.038381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a8324-4b0c-4d6a-b5d9-608c69530414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.038649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a8324-4b0c-4d6a-b5d9-608c69530414 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.039280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a8324-4b0c-4d6a-b5d9-608c69530414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.041529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a8324-4b0c-4d6a-b5d9-608c69530414 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.042286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e63008-fc22-489b-aec0-1c23ccdda4ef admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.047632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.048332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.048614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.048890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] PATH_INFO: `/v3/users/6e3802355e7c4a099d9d8659b431f4ae` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.050062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3802355e7c4a099d9d8659b431f4ae)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.053197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.078745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1029/2057] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:28 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:29.080304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a8324-4b0c-4d6a-b5d9-608c69530414 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:29.093152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a8324-4b0c-4d6a-b5d9-608c69530414 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.101675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bc241-3306-4761-a6fd-a2b74097acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.102458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bc241-3306-4761-a6fd-a2b74097acba admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.102959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bc241-3306-4761-a6fd-a2b74097acba admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.103335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bc241-3306-4761-a6fd-a2b74097acba admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.104813 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bc241-3306-4761-a6fd-a2b74097acba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.106753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bc241-3306-4761-a6fd-a2b74097acba admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.119681 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1030/2058] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:29.121782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3802355e7c4a099d9d8659b431f4ae'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:29.122884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bc241-3306-4761-a6fd-a2b74097acba admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:29.133396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3802355e7c4a099d9d8659b431f4ae'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:29.134474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3802355e7c4a099d9d8659b431f4ae'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:29.137476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bc241-3306-4761-a6fd-a2b74097acba admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.142952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.143829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.144363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.144738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] PATH_INFO: `/v3/projects/ba1ef7b3162244ef98793729742fa9dc/users/b14d74d4554944b4b3806f454ad0fa7e/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.152746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3802355e7c4a099d9d8659b431f4ae'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:29.153531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a4c6ab95-7da0-452f-98ba-02b8324b4ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3802355e7c4a099d9d8659b431f4ae'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:29.163422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1ef7b3162244ef98793729742fa9dc, user_id=b14d74d4554944b4b3806f454ad0fa7e, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.166612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.172784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1029/2059] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:28 2022] DELETE /identity/v3/users/6e3802355e7c4a099d9d8659b431f4ae => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:29.188327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddbece-a5d5-45ca-bc11-2f49d91818f6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.188327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddbece-a5d5-45ca-bc11-2f49d91818f6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.188327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddbece-a5d5-45ca-bc11-2f49d91818f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.196795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1031/2060] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:29 2022] PUT /identity/v3/projects/ba1ef7b3162244ef98793729742fa9dc/users/b14d74d4554944b4b3806f454ad0fa7e/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:29.200580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:29.202263 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:29.202696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b2ddbece-a5d5-45ca-bc11-2f49d91818f6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:29.203967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b2ddbece-a5d5-45ca-bc11-2f49d91818f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:29.275940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1030/2061] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:29.284134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb7260-8c31-4845-ad73-ceab615ea52b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.285453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddbece-a5d5-45ca-bc11-2f49d91818f6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:29.287008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b2ddbece-a5d5-45ca-bc11-2f49d91818f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:29.289680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc946b8-4001-4871-9b96-49f85d9abb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.290415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc946b8-4001-4871-9b96-49f85d9abb48 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.290759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc946b8-4001-4871-9b96-49f85d9abb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.291166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc946b8-4001-4871-9b96-49f85d9abb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.291822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc946b8-4001-4871-9b96-49f85d9abb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.293535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc946b8-4001-4871-9b96-49f85d9abb48 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.305107 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1032/2062] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:29:29.309598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc946b8-4001-4871-9b96-49f85d9abb48 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:29.321024 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc946b8-4001-4871-9b96-49f85d9abb48 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.321334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:29.325961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d63283-c57f-41bd-ad86-8145000dcd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.326686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63283-c57f-41bd-ad86-8145000dcd41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.327043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63283-c57f-41bd-ad86-8145000dcd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.327376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d63283-c57f-41bd-ad86-8145000dcd41 admin admin] PATH_INFO: `/v3/projects/ba1ef7b3162244ef98793729742fa9dc/users/b14d74d4554944b4b3806f454ad0fa7e/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.329450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d63283-c57f-41bd-ad86-8145000dcd41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1ef7b3162244ef98793729742fa9dc, user_id=b14d74d4554944b4b3806f454ad0fa7e, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.332210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d63283-c57f-41bd-ad86-8145000dcd41 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.355657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1033/2063] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:29 2022] PUT /identity/v3/projects/ba1ef7b3162244ef98793729742fa9dc/users/b14d74d4554944b4b3806f454ad0fa7e/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:29.357129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbca95b-84cc-453e-9f40-e0ffd4bb4b82 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.357414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbca95b-84cc-453e-9f40-e0ffd4bb4b82 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.357682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbca95b-84cc-453e-9f40-e0ffd4bb4b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.366593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddbece-a5d5-45ca-bc11-2f49d91818f6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.388805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-8bbca95b-84cc-453e-9f40-e0ffd4bb4b82 None None] MFA Rules not processed for user `b14d74d4554944b4b3806f454ad0fa7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:29.390439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8bbca95b-84cc-453e-9f40-e0ffd4bb4b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:29.397376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:29.434900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e0df3-c3fd-4ce5-a9f8-259bddd06f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.435780 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e0df3-c3fd-4ce5-a9f8-259bddd06f30 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.436248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e0df3-c3fd-4ce5-a9f8-259bddd06f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.436672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e0df3-c3fd-4ce5-a9f8-259bddd06f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.473396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1034/2064] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:29.485782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbca95b-84cc-453e-9f40-e0ffd4bb4b82 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:29.487218 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8bbca95b-84cc-453e-9f40-e0ffd4bb4b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:29.512800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e0df3-c3fd-4ce5-a9f8-259bddd06f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.514244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e0df3-c3fd-4ce5-a9f8-259bddd06f30 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.526167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1031/2065] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:29:29.554368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbca95b-84cc-453e-9f40-e0ffd4bb4b82 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.558548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.559405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.560347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.560698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.572747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.574311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.586886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1035/2066] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:29.678000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e0df3-c3fd-4ce5-a9f8-259bddd06f30 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:29.688314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e0df3-c3fd-4ce5-a9f8-259bddd06f30 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:29.693533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5ad4e-bf9b-424f-8e1e-05fe5976c9c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:29.694405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5ad4e-bf9b-424f-8e1e-05fe5976c9c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:29.694931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5ad4e-bf9b-424f-8e1e-05fe5976c9c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:29.695283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5ad4e-bf9b-424f-8e1e-05fe5976c9c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:29.701190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-11e5ad4e-bf9b-424f-8e1e-05fe5976c9c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:29.777449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5ad4e-bf9b-424f-8e1e-05fe5976c9c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:29.778930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5ad4e-bf9b-424f-8e1e-05fe5976c9c0 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:29.790336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1032/2067] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:30.055051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:30.056888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:30.071028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25628c-c6de-44bf-bb8f-935e1d2a4a2f service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:30.075530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:30.076189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.076468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.076736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] PATH_INFO: `/v3/projects/ea2bfe1166f74bb290425cd93ea4e333` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.077839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea2bfe1166f74bb290425cd93ea4e333)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:30.079749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:30.131622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea2bfe1166f74bb290425cd93ea4e333', 'request_id': 'req-2c64474f-3a7f-4785-b69d-339b8f212ea3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:30.138072 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea2bfe1166f74bb290425cd93ea4e333', 'request_id': 'req-2c64474f-3a7f-4785-b69d-339b8f212ea3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:30.144198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea2bfe1166f74bb290425cd93ea4e333', 'request_id': 'req-2c64474f-3a7f-4785-b69d-339b8f212ea3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:30.160179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] The token cache is being invalidate because project ea2bfe1166f74bb290425cd93ea4e333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:30.160471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:30.161838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-2c64474f-3a7f-4785-b69d-339b8f212ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:30.163473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1036/2068] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:30 2022] DELETE /identity/v3/projects/ea2bfe1166f74bb290425cd93ea4e333 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:30.168637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588bebb-8674-404d-858d-018d47b25aad None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.169143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588bebb-8674-404d-858d-018d47b25aad None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.169257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7588bebb-8674-404d-858d-018d47b25aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.189577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7588bebb-8674-404d-858d-018d47b25aad None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:30.190887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7588bebb-8674-404d-858d-018d47b25aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:30.221825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:30.272537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1033/2069] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:30.281211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c5fa-985a-4611-b517-b165535fe74d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.281491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c5fa-985a-4611-b517-b165535fe74d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.281801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4c5fa-985a-4611-b517-b165535fe74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.320148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c9b4c5fa-985a-4611-b517-b165535fe74d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:30.320148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c9b4c5fa-985a-4611-b517-b165535fe74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:30.332145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1037/2070] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:30.334681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588bebb-8674-404d-858d-018d47b25aad None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:30.344467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7588bebb-8674-404d-858d-018d47b25aad None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:30.350703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:30.351395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.351735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.352068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.352823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:30.353205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:30.354915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:30.373612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1034/2071] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:29:30.377812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4c5fa-985a-4611-b517-b165535fe74d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:30.388219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4c5fa-985a-4611-b517-b165535fe74d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:30.392982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d320d7-8bd7-46d1-86aa-8c9fcb30a816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:30.393652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d320d7-8bd7-46d1-86aa-8c9fcb30a816 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.394000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d320d7-8bd7-46d1-86aa-8c9fcb30a816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.394340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d320d7-8bd7-46d1-86aa-8c9fcb30a816 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.395044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d320d7-8bd7-46d1-86aa-8c9fcb30a816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:30.397080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d320d7-8bd7-46d1-86aa-8c9fcb30a816 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:30.420522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:30.433928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1038/2072] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:30 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:30.435178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:30.446382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96b687-fe57-4527-93f9-eac014e6b9e3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:30.506045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2a5e9-d5b6-4e40-87cf-83e3ed296f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:30.506654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a5e9-d5b6-4e40-87cf-83e3ed296f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.506932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a5e9-d5b6-4e40-87cf-83e3ed296f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.507199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a5e9-d5b6-4e40-87cf-83e3ed296f0a admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.507822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2a5e9-d5b6-4e40-87cf-83e3ed296f0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:30.509759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2a5e9-d5b6-4e40-87cf-83e3ed296f0a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:30.530971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1035/2073] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:30 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:30.534947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d320d7-8bd7-46d1-86aa-8c9fcb30a816 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:30.546449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d320d7-8bd7-46d1-86aa-8c9fcb30a816 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:30.551033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde591e-2561-4d31-befe-0b17f7ea81cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:30.551622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde591e-2561-4d31-befe-0b17f7ea81cf admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.552467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde591e-2561-4d31-befe-0b17f7ea81cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.552739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde591e-2561-4d31-befe-0b17f7ea81cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.553529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde591e-2561-4d31-befe-0b17f7ea81cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:30.555102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde591e-2561-4d31-befe-0b17f7ea81cf admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:30.565471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1039/2074] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:30.570189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2a5e9-d5b6-4e40-87cf-83e3ed296f0a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:30.579952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2a5e9-d5b6-4e40-87cf-83e3ed296f0a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:30.585746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da555a-0015-434d-a212-c42ba9c22930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:30.586343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da555a-0015-434d-a212-c42ba9c22930 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.586619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da555a-0015-434d-a212-c42ba9c22930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.586889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da555a-0015-434d-a212-c42ba9c22930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.587527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da555a-0015-434d-a212-c42ba9c22930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:30.589139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da555a-0015-434d-a212-c42ba9c22930 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:30.601906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1036/2075] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:29:30.605972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde591e-2561-4d31-befe-0b17f7ea81cf admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:30.616345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde591e-2561-4d31-befe-0b17f7ea81cf admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:30.621272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:30.621849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.622134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.622402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] PATH_INFO: `/v3/projects/ab0c02c8d1fa4a9fbbb64a655643ff31/users/ea2b7595ca6d446c84e95a05a701e9dc/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.638815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0c02c8d1fa4a9fbbb64a655643ff31, user_id=ea2b7595ca6d446c84e95a05a701e9dc, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:30.641525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:30.656228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1040/2076] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:30 2022] PUT /identity/v3/projects/ab0c02c8d1fa4a9fbbb64a655643ff31/users/ea2b7595ca6d446c84e95a05a701e9dc/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:30.661303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114dd4c-3062-4473-b367-875929e7ac5f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.661662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114dd4c-3062-4473-b367-875929e7ac5f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.661933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114dd4c-3062-4473-b367-875929e7ac5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.681043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7114dd4c-3062-4473-b367-875929e7ac5f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:30.682505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7114dd4c-3062-4473-b367-875929e7ac5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:30.749390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1037/2077] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:30.759735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:30.761481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:30.822051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eadc6-7ad1-4a99-a54c-54271860b455 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:30.825910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059d94-03a0-4e28-ac2b-616cddea6965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:30.826599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059d94-03a0-4e28-ac2b-616cddea6965 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:30.826947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059d94-03a0-4e28-ac2b-616cddea6965 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:30.827299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059d94-03a0-4e28-ac2b-616cddea6965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:30.837914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059d94-03a0-4e28-ac2b-616cddea6965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:30.839356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059d94-03a0-4e28-ac2b-616cddea6965 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:30.849624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1041/2078] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:35.840429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b4ecf-ecda-4c6b-b6f7-7208ed705277 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:35.841131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b4ecf-ecda-4c6b-b6f7-7208ed705277 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:35.841131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b4ecf-ecda-4c6b-b6f7-7208ed705277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:35.894996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-881b4ecf-ecda-4c6b-b6f7-7208ed705277 None None] MFA Rules not processed for user `ea2b7595ca6d446c84e95a05a701e9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:35.896591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-881b4ecf-ecda-4c6b-b6f7-7208ed705277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:35.927281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:35.992276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1038/2079] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:35.993303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059d94-03a0-4e28-ac2b-616cddea6965 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:36.005454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059d94-03a0-4e28-ac2b-616cddea6965 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:36.014766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362f939-c99e-4772-9b14-a16826ca48ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:36.015386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f939-c99e-4772-9b14-a16826ca48ec admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:36.015660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f939-c99e-4772-9b14-a16826ca48ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:36.015928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f939-c99e-4772-9b14-a16826ca48ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:36.016641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362f939-c99e-4772-9b14-a16826ca48ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:36.018613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362f939-c99e-4772-9b14-a16826ca48ec admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:36.065531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1042/2080] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:35 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:36.069986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b4ecf-ecda-4c6b-b6f7-7208ed705277 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:36.082488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b4ecf-ecda-4c6b-b6f7-7208ed705277 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:36.151919 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974a611-70af-4980-a40e-a73cfc204f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:36.152680 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974a611-70af-4980-a40e-a73cfc204f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:36.153018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974a611-70af-4980-a40e-a73cfc204f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:36.153349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974a611-70af-4980-a40e-a73cfc204f30 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:36.154041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974a611-70af-4980-a40e-a73cfc204f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:36.156065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974a611-70af-4980-a40e-a73cfc204f30 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:36.183763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1039/2081] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:36 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:36.187352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362f939-c99e-4772-9b14-a16826ca48ec admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:36.199454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362f939-c99e-4772-9b14-a16826ca48ec admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:36.204468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d4f18-b147-4c9e-be54-19acf5c31d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:36.205161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d4f18-b147-4c9e-be54-19acf5c31d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:36.205527 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d4f18-b147-4c9e-be54-19acf5c31d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:36.205868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d4f18-b147-4c9e-be54-19acf5c31d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:36.206583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d4f18-b147-4c9e-be54-19acf5c31d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:36.208925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d4f18-b147-4c9e-be54-19acf5c31d72 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:36.220920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1043/2082] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:36.226042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974a611-70af-4980-a40e-a73cfc204f30 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:36.240554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974a611-70af-4980-a40e-a73cfc204f30 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:36.245934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805041-fbbb-4302-8fed-8a6a06d39e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:36.246662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805041-fbbb-4302-8fed-8a6a06d39e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:36.246946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805041-fbbb-4302-8fed-8a6a06d39e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:36.247149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805041-fbbb-4302-8fed-8a6a06d39e9b admin admin] PATH_INFO: `/v3/projects/143280ba084940e0b9bb18410615a201/users/ba8034ff9f7f4af89a68c59e72ce02d4/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:36.274160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56805041-fbbb-4302-8fed-8a6a06d39e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=143280ba084940e0b9bb18410615a201, user_id=ba8034ff9f7f4af89a68c59e72ce02d4, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:36.277046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56805041-fbbb-4302-8fed-8a6a06d39e9b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:36.291894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1040/2083] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:36 2022] PUT /identity/v3/projects/143280ba084940e0b9bb18410615a201/users/ba8034ff9f7f4af89a68c59e72ce02d4/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:36.295793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d4f18-b147-4c9e-be54-19acf5c31d72 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:36.308254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d4f18-b147-4c9e-be54-19acf5c31d72 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:36.371215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f8caf-0d7c-4607-9a1e-ff25d9791c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:36.371850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f8caf-0d7c-4607-9a1e-ff25d9791c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:36.372115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f8caf-0d7c-4607-9a1e-ff25d9791c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:36.372392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f8caf-0d7c-4607-9a1e-ff25d9791c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:36.373070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f8caf-0d7c-4607-9a1e-ff25d9791c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:36.374664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f8caf-0d7c-4607-9a1e-ff25d9791c31 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:36.386712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1044/2084] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:29:36.391056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805041-fbbb-4302-8fed-8a6a06d39e9b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:36.403026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805041-fbbb-4302-8fed-8a6a06d39e9b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:36.409071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:36.409753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:36.410029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:36.410297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] PATH_INFO: `/v3/projects/143280ba084940e0b9bb18410615a201/users/ba8034ff9f7f4af89a68c59e72ce02d4/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:36.412992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] RBAC: Authorizing `identity:create_grant(project_id=143280ba084940e0b9bb18410615a201, user_id=ba8034ff9f7f4af89a68c59e72ce02d4, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:36.415774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:36.435285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1041/2085] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:36 2022] PUT /identity/v3/projects/143280ba084940e0b9bb18410615a201/users/ba8034ff9f7f4af89a68c59e72ce02d4/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:41.868216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc59a6-6e34-4283-b870-725bae911bda None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:41.868693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc59a6-6e34-4283-b870-725bae911bda None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:41.868938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc59a6-6e34-4283-b870-725bae911bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:41.904380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-9fcc59a6-6e34-4283-b870-725bae911bda None None] MFA Rules not processed for user `ba8034ff9f7f4af89a68c59e72ce02d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:41.907446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9fcc59a6-6e34-4283-b870-725bae911bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:41.934678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:42.022290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1045/2086] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:42.038745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:42.040880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:42.174842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fe8c-79e0-473f-b5de-a2a20411530d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:42.181774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3403a4a-3dfa-4ca5-9af5-fdf663feb477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:42.182453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403a4a-3dfa-4ca5-9af5-fdf663feb477 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:42.182795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403a4a-3dfa-4ca5-9af5-fdf663feb477 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:42.183350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3403a4a-3dfa-4ca5-9af5-fdf663feb477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:42.197825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:42.256777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3403a4a-3dfa-4ca5-9af5-fdf663feb477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:42.258506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3403a4a-3dfa-4ca5-9af5-fdf663feb477 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:42.268142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1042/2087] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:46.222063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc59a6-6e34-4283-b870-725bae911bda None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:46.224502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9fcc59a6-6e34-4283-b870-725bae911bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:46.302608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc59a6-6e34-4283-b870-725bae911bda None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:46.307926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:46.308677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:46.309159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:46.309507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] PATH_INFO: `/v3/users/8cc4f4a8284d4112a675bd8ef608e9e8` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:46.310719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc4f4a8284d4112a675bd8ef608e9e8)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:46.312801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:46.367405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc4f4a8284d4112a675bd8ef608e9e8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:46.380194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc4f4a8284d4112a675bd8ef608e9e8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:46.380909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc4f4a8284d4112a675bd8ef608e9e8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:46.387623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc4f4a8284d4112a675bd8ef608e9e8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:46.394573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc4f4a8284d4112a675bd8ef608e9e8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:46.401054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1046/2088] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:46 2022] DELETE /identity/v3/users/8cc4f4a8284d4112a675bd8ef608e9e8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:46.405899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8615be-e7f8-4063-a2ab-b62f647ba5cf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:46.406407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8615be-e7f8-4063-a2ab-b62f647ba5cf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:46.406719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8615be-e7f8-4063-a2ab-b62f647ba5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:46.426528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7c8615be-e7f8-4063-a2ab-b62f647ba5cf None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:46.428127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7c8615be-e7f8-4063-a2ab-b62f647ba5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:46.491775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1043/2089] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:46.501915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:46.503329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:46.556375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:46.576278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cec812-025d-48f3-8d89-c492403286c2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:46.584289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a14e8-edd8-425c-81f0-32e1c1969d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:46.584289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a14e8-edd8-425c-81f0-32e1c1969d41 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:46.584289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a14e8-edd8-425c-81f0-32e1c1969d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:46.584289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a14e8-edd8-425c-81f0-32e1c1969d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:46.591806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a14e8-edd8-425c-81f0-32e1c1969d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:46.593438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a14e8-edd8-425c-81f0-32e1c1969d41 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:46.604203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1047/2090] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:29:46.907744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8615be-e7f8-4063-a2ab-b62f647ba5cf None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:46.909847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7c8615be-e7f8-4063-a2ab-b62f647ba5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:46.933381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8615be-e7f8-4063-a2ab-b62f647ba5cf None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:46.941446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:46.942593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:46.943021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:46.943567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] PATH_INFO: `/v3/projects/9518076a0c7d4c15919f7c1ffda446be` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:46.945382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9518076a0c7d4c15919f7c1ffda446be)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:46.947444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:47.001129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9518076a0c7d4c15919f7c1ffda446be', 'request_id': 'req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.009014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9518076a0c7d4c15919f7c1ffda446be', 'request_id': 'req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.015488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9518076a0c7d4c15919f7c1ffda446be', 'request_id': 'req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.027072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] The token cache is being invalidate because project 9518076a0c7d4c15919f7c1ffda446be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:47.027379 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.028046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.029837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1044/2091] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:46 2022] DELETE /identity/v3/projects/9518076a0c7d4c15919f7c1ffda446be => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:47.033797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a14e8-edd8-425c-81f0-32e1c1969d41 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:47.051748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a14e8-edd8-425c-81f0-32e1c1969d41 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:47.115096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:47.116743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:47.117019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:47.117287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] PATH_INFO: `/v3/users/b14d74d4554944b4b3806f454ad0fa7e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:47.119552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b14d74d4554944b4b3806f454ad0fa7e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:47.123304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:47.182770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b14d74d4554944b4b3806f454ad0fa7e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.198661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b14d74d4554944b4b3806f454ad0fa7e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.199766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b14d74d4554944b4b3806f454ad0fa7e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.212592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b14d74d4554944b4b3806f454ad0fa7e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.220557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e32d973-627d-4953-a43c-3d5752bc9ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b14d74d4554944b4b3806f454ad0fa7e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.227166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1048/2092] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:47 2022] DELETE /identity/v3/users/b14d74d4554944b4b3806f454ad0fa7e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:47.663537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:47.665335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:47.681435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c7fa4-d648-4bc4-b4bc-64c2aa1852cb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:47.737722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:47.766732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:47.767531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:47.767874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:47.768256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] PATH_INFO: `/v3/projects/ba1ef7b3162244ef98793729742fa9dc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:47.769454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba1ef7b3162244ef98793729742fa9dc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:47.772762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:47.825498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1ef7b3162244ef98793729742fa9dc', 'request_id': 'req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.834023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1ef7b3162244ef98793729742fa9dc', 'request_id': 'req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.840006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1ef7b3162244ef98793729742fa9dc', 'request_id': 'req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.850953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] The token cache is being invalidate because project ba1ef7b3162244ef98793729742fa9dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:47.851237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.851856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3cb78fed-9f81-4b1c-a30d-e365b9c8cb1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:47.853354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1045/2093] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:47 2022] DELETE /identity/v3/projects/ba1ef7b3162244ef98793729742fa9dc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:47.858283 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce239cc-363d-4986-8b1a-a15b6e2fe91b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:47.858571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce239cc-363d-4986-8b1a-a15b6e2fe91b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:47.858870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce239cc-363d-4986-8b1a-a15b6e2fe91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:47.880490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4ce239cc-363d-4986-8b1a-a15b6e2fe91b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:47.881673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4ce239cc-363d-4986-8b1a-a15b6e2fe91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:47.930896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:47.945161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1049/2094] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:47.952771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186f45-e8c9-411c-b7fb-cb306ae31c60 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:47.953131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186f45-e8c9-411c-b7fb-cb306ae31c60 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:47.953478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186f45-e8c9-411c-b7fb-cb306ae31c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:47.973932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6f186f45-e8c9-411c-b7fb-cb306ae31c60 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:47.975145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6f186f45-e8c9-411c-b7fb-cb306ae31c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:47.983028 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1046/2095] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:47.987370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce239cc-363d-4986-8b1a-a15b6e2fe91b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:47.997160 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce239cc-363d-4986-8b1a-a15b6e2fe91b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.002005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.002608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.002884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.003151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.003798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:48.004115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.006484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.017902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1050/2096] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:29:48.022940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f186f45-e8c9-411c-b7fb-cb306ae31c60 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.033216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f186f45-e8c9-411c-b7fb-cb306ae31c60 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.039635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5287ec-a8fc-4ee0-a4b2-ceb5258b8246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.040290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5287ec-a8fc-4ee0-a4b2-ceb5258b8246 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.040568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5287ec-a8fc-4ee0-a4b2-ceb5258b8246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.040835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5287ec-a8fc-4ee0-a4b2-ceb5258b8246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.041474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5287ec-a8fc-4ee0-a4b2-ceb5258b8246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.043368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5287ec-a8fc-4ee0-a4b2-ceb5258b8246 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.071952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1047/2097] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:48 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:48.076153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.086868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4ed1c-25b4-4b96-99b6-ee446fb2be23 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.144689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2573a-1617-4e67-aeb9-892c505f8091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.146566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2573a-1617-4e67-aeb9-892c505f8091 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.146839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2573a-1617-4e67-aeb9-892c505f8091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.147106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2573a-1617-4e67-aeb9-892c505f8091 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.147751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2573a-1617-4e67-aeb9-892c505f8091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.149688 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2573a-1617-4e67-aeb9-892c505f8091 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.172426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1051/2098] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:48 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:48.176816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5287ec-a8fc-4ee0-a4b2-ceb5258b8246 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.189124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5287ec-a8fc-4ee0-a4b2-ceb5258b8246 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.194966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92737f48-8a7a-4da2-aed4-7c7016398937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.195609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92737f48-8a7a-4da2-aed4-7c7016398937 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.195914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92737f48-8a7a-4da2-aed4-7c7016398937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.196228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92737f48-8a7a-4da2-aed4-7c7016398937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.196875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92737f48-8a7a-4da2-aed4-7c7016398937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.198468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92737f48-8a7a-4da2-aed4-7c7016398937 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.209639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1048/2099] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:48.213669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2573a-1617-4e67-aeb9-892c505f8091 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.223931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2573a-1617-4e67-aeb9-892c505f8091 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.228475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2446d67-2430-4714-95c0-f514d6eb39af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.229076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2446d67-2430-4714-95c0-f514d6eb39af admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.229341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2446d67-2430-4714-95c0-f514d6eb39af admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.229606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2446d67-2430-4714-95c0-f514d6eb39af admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.230241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2446d67-2430-4714-95c0-f514d6eb39af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.231820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2446d67-2430-4714-95c0-f514d6eb39af admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.247520 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1052/2100] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:48.251784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92737f48-8a7a-4da2-aed4-7c7016398937 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.262517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92737f48-8a7a-4da2-aed4-7c7016398937 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.267184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f360-99ac-4a37-aed9-9edffde18601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.267826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f360-99ac-4a37-aed9-9edffde18601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.268131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f360-99ac-4a37-aed9-9edffde18601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.268410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899f360-99ac-4a37-aed9-9edffde18601 admin admin] PATH_INFO: `/v3/projects/aadc7fe2e55f4c22a2cadfbb5c899a63/users/1fd2f5ca8cdc4e37941328df0279f8a1/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.283584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899f360-99ac-4a37-aed9-9edffde18601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aadc7fe2e55f4c22a2cadfbb5c899a63, user_id=1fd2f5ca8cdc4e37941328df0279f8a1, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.286447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899f360-99ac-4a37-aed9-9edffde18601 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.299433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1049/2101] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:48 2022] PUT /identity/v3/projects/aadc7fe2e55f4c22a2cadfbb5c899a63/users/1fd2f5ca8cdc4e37941328df0279f8a1/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:48.311066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add835f3-4679-4bba-82fc-4f465a6805c1 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.311351 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add835f3-4679-4bba-82fc-4f465a6805c1 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.311625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add835f3-4679-4bba-82fc-4f465a6805c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.339730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-add835f3-4679-4bba-82fc-4f465a6805c1 None None] MFA Rules not processed for user `1fd2f5ca8cdc4e37941328df0279f8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:48.341365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-add835f3-4679-4bba-82fc-4f465a6805c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:48.407867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1053/2102] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:48.412482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f360-99ac-4a37-aed9-9edffde18601 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.479524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899f360-99ac-4a37-aed9-9edffde18601 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.484310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025809e-e7d9-41c1-acdc-a03db1bd6268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.484922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025809e-e7d9-41c1-acdc-a03db1bd6268 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.485200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025809e-e7d9-41c1-acdc-a03db1bd6268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.485470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025809e-e7d9-41c1-acdc-a03db1bd6268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.486101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025809e-e7d9-41c1-acdc-a03db1bd6268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.487983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025809e-e7d9-41c1-acdc-a03db1bd6268 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.493991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:48.519002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1050/2103] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:48 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:48.521712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add835f3-4679-4bba-82fc-4f465a6805c1 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.532629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add835f3-4679-4bba-82fc-4f465a6805c1 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.589693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:48.598809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a6207-f0f4-4b30-ba64-88de50dce20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.599557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6207-f0f4-4b30-ba64-88de50dce20b admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.599841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6207-f0f4-4b30-ba64-88de50dce20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.600159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a6207-f0f4-4b30-ba64-88de50dce20b admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.600832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a6207-f0f4-4b30-ba64-88de50dce20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.602750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a6207-f0f4-4b30-ba64-88de50dce20b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.623559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1054/2104] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:48 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:48.627548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025809e-e7d9-41c1-acdc-a03db1bd6268 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.638213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025809e-e7d9-41c1-acdc-a03db1bd6268 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.642861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b15db-c9f7-4051-b832-27b71d4fed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.643459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b15db-c9f7-4051-b832-27b71d4fed60 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.643738 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b15db-c9f7-4051-b832-27b71d4fed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.644008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b15db-c9f7-4051-b832-27b71d4fed60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.644688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b15db-c9f7-4051-b832-27b71d4fed60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.646392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b15db-c9f7-4051-b832-27b71d4fed60 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.656121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1051/2105] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:29:48.659875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a6207-f0f4-4b30-ba64-88de50dce20b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.670377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a6207-f0f4-4b30-ba64-88de50dce20b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.674939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30378575-72e3-43ec-8ea4-122c79422733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.675574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30378575-72e3-43ec-8ea4-122c79422733 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.675853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30378575-72e3-43ec-8ea4-122c79422733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.676790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30378575-72e3-43ec-8ea4-122c79422733 admin admin] PATH_INFO: `/v3/projects/2aa232fa8b654f7181a17fe975227681/users/f56be60cd76e42b69deac5b2d2d54263/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.692665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30378575-72e3-43ec-8ea4-122c79422733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa232fa8b654f7181a17fe975227681, user_id=f56be60cd76e42b69deac5b2d2d54263, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.696376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30378575-72e3-43ec-8ea4-122c79422733 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.710416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1055/2106] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:48 2022] PUT /identity/v3/projects/2aa232fa8b654f7181a17fe975227681/users/f56be60cd76e42b69deac5b2d2d54263/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:48.714400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b15db-c9f7-4051-b832-27b71d4fed60 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.725675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b15db-c9f7-4051-b832-27b71d4fed60 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.782286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5269933-599b-413e-9e2e-00c3da38222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.782901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5269933-599b-413e-9e2e-00c3da38222b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.783173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5269933-599b-413e-9e2e-00c3da38222b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.783438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5269933-599b-413e-9e2e-00c3da38222b admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.784197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5269933-599b-413e-9e2e-00c3da38222b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.785821 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5269933-599b-413e-9e2e-00c3da38222b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.796479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1052/2107] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:48.800637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30378575-72e3-43ec-8ea4-122c79422733 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.811588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30378575-72e3-43ec-8ea4-122c79422733 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:48.817192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:48.817952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.818236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.818502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] PATH_INFO: `/v3/projects/2aa232fa8b654f7181a17fe975227681/users/f56be60cd76e42b69deac5b2d2d54263/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.820456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa232fa8b654f7181a17fe975227681, user_id=f56be60cd76e42b69deac5b2d2d54263, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:48.823079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:48.834563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1056/2108] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:48 2022] PUT /identity/v3/projects/2aa232fa8b654f7181a17fe975227681/users/f56be60cd76e42b69deac5b2d2d54263/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:48.846728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ecad1-7c49-4b1a-b62d-8d09c6843c7f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:48.847081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ecad1-7c49-4b1a-b62d-8d09c6843c7f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:48.847306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ecad1-7c49-4b1a-b62d-8d09c6843c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:48.878249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4b2ecad1-7c49-4b1a-b62d-8d09c6843c7f None None] MFA Rules not processed for user `f56be60cd76e42b69deac5b2d2d54263`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:48.880251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b2ecad1-7c49-4b1a-b62d-8d09c6843c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:48.959587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1053/2109] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:48.971189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:48.973365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:49.033533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039607b-1e7d-444b-83fc-febbbe5e507f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:49.037342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0de450-3eb1-4fe3-84b6-9780229abc9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:49.037945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0de450-3eb1-4fe3-84b6-9780229abc9b service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:49.038225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0de450-3eb1-4fe3-84b6-9780229abc9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:49.038495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0de450-3eb1-4fe3-84b6-9780229abc9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:49.047480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0de450-3eb1-4fe3-84b6-9780229abc9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:49.048948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0de450-3eb1-4fe3-84b6-9780229abc9b service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:49.059552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1057/2110] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:49.155222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ecad1-7c49-4b1a-b62d-8d09c6843c7f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:49.167189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ecad1-7c49-4b1a-b62d-8d09c6843c7f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:49.171370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:49.172151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:49.172542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:49.172924 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:49.236344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:49.240017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:49.241795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:49.252871 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1054/2111] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:52.094934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb015-3097-4308-af21-b70f2569c35c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:52.095388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb015-3097-4308-af21-b70f2569c35c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:52.095476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb015-3097-4308-af21-b70f2569c35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:52.117001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-0f7bb015-3097-4308-af21-b70f2569c35c None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:52.118809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0f7bb015-3097-4308-af21-b70f2569c35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:52.179059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1058/2112] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:29:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:52.223103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:52.224541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:52.297667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61c55-54f1-4f04-bf9d-bb4aac159cf7 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:52.303355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a663a0-40b2-4639-a96b-5feafc83652f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:52.303971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a663a0-40b2-4639-a96b-5feafc83652f service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:52.304280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a663a0-40b2-4639-a96b-5feafc83652f service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:52.304556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a663a0-40b2-4639-a96b-5feafc83652f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:52.314950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a663a0-40b2-4639-a96b-5feafc83652f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:52.317231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a663a0-40b2-4639-a96b-5feafc83652f service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:52.332915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1055/2113] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:29:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:29:55.292824 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb015-3097-4308-af21-b70f2569c35c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:55.354621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:55.370173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb015-3097-4308-af21-b70f2569c35c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:55.380759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:55.381658 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:55.382015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:55.382353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] PATH_INFO: `/v3/users/1fd2f5ca8cdc4e37941328df0279f8a1` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:55.383664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd2f5ca8cdc4e37941328df0279f8a1)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:55.385746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:55.442930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd2f5ca8cdc4e37941328df0279f8a1'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:55.455126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd2f5ca8cdc4e37941328df0279f8a1'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:55.455829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd2f5ca8cdc4e37941328df0279f8a1'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:55.464375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd2f5ca8cdc4e37941328df0279f8a1'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:55.472742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd2f5ca8cdc4e37941328df0279f8a1'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:55.480063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1059/2114] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:55 2022] DELETE /identity/v3/users/1fd2f5ca8cdc4e37941328df0279f8a1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:55.484746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e36c8c-fbbb-47e8-8fd1-778b7b9ad8ff None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:55.485270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e36c8c-fbbb-47e8-8fd1-778b7b9ad8ff None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:55.485270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e36c8c-fbbb-47e8-8fd1-778b7b9ad8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:55.509288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b5e36c8c-fbbb-47e8-8fd1-778b7b9ad8ff None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:55.510984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b5e36c8c-fbbb-47e8-8fd1-778b7b9ad8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:55.578473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1056/2115] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:55.589576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:55.591082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:55.653327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74635bec-7ac6-4102-8726-f30282c78188 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:55.657206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85650e-44ab-4bd8-b3e7-2872ec302f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:55.657831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85650e-44ab-4bd8-b3e7-2872ec302f3a service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:55.658169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85650e-44ab-4bd8-b3e7-2872ec302f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:55.658447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85650e-44ab-4bd8-b3e7-2872ec302f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:55.669122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85650e-44ab-4bd8-b3e7-2872ec302f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:55.670875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85650e-44ab-4bd8-b3e7-2872ec302f3a service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:55.682348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1060/2116] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:56.183267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e36c8c-fbbb-47e8-8fd1-778b7b9ad8ff None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:56.184790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b5e36c8c-fbbb-47e8-8fd1-778b7b9ad8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:56.201248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e36c8c-fbbb-47e8-8fd1-778b7b9ad8ff None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:56.217422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:56.218259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:56.219416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:56.219886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] PATH_INFO: `/v3/projects/aadc7fe2e55f4c22a2cadfbb5c899a63` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:56.221785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aadc7fe2e55f4c22a2cadfbb5c899a63)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:56.225381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:56.304549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aadc7fe2e55f4c22a2cadfbb5c899a63', 'request_id': 'req-645f3485-7f7a-460d-a6a6-04aecee97ad7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.314723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aadc7fe2e55f4c22a2cadfbb5c899a63', 'request_id': 'req-645f3485-7f7a-460d-a6a6-04aecee97ad7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.326257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aadc7fe2e55f4c22a2cadfbb5c899a63', 'request_id': 'req-645f3485-7f7a-460d-a6a6-04aecee97ad7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.349624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] The token cache is being invalidate because project aadc7fe2e55f4c22a2cadfbb5c899a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:56.349994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.350579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.352350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1057/2117] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:56 2022] DELETE /identity/v3/projects/aadc7fe2e55f4c22a2cadfbb5c899a63 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:56.356219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85650e-44ab-4bd8-b3e7-2872ec302f3a service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:56.372452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85650e-44ab-4bd8-b3e7-2872ec302f3a service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:56.467685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:56.468591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:56.469038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:56.469473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] PATH_INFO: `/v3/users/f56be60cd76e42b69deac5b2d2d54263` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:56.470608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56be60cd76e42b69deac5b2d2d54263)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:56.472578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:56.495988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:56.544458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56be60cd76e42b69deac5b2d2d54263'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.565665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56be60cd76e42b69deac5b2d2d54263'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.566513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56be60cd76e42b69deac5b2d2d54263'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.573867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56be60cd76e42b69deac5b2d2d54263'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.581470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-359b741a-9894-474f-ae64-8e2a8645c945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56be60cd76e42b69deac5b2d2d54263'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:56.609264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1061/2118] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:56 2022] DELETE /identity/v3/users/f56be60cd76e42b69deac5b2d2d54263 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:57.158405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:57.161543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:57.181489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f3485-7f7a-460d-a6a6-04aecee97ad7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:57.187090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:57.305020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:57.306465 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.306878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.307152 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] PATH_INFO: `/v3/projects/2aa232fa8b654f7181a17fe975227681` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.308527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa232fa8b654f7181a17fe975227681)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:57.310607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:57.368787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa232fa8b654f7181a17fe975227681', 'request_id': 'req-a40930c4-b610-438e-b70e-649057249660'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:57.398254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa232fa8b654f7181a17fe975227681', 'request_id': 'req-a40930c4-b610-438e-b70e-649057249660'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:57.405971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa232fa8b654f7181a17fe975227681', 'request_id': 'req-a40930c4-b610-438e-b70e-649057249660'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:57.417312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] The token cache is being invalidate because project 2aa232fa8b654f7181a17fe975227681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:29:57.417687 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:57.418435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a40930c4-b610-438e-b70e-649057249660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:57.420166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1058/2119] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:29:57 2022] DELETE /identity/v3/projects/2aa232fa8b654f7181a17fe975227681 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:57.424990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6989d-32c7-4640-bc7a-f68c3812dcb7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.425516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6989d-32c7-4640-bc7a-f68c3812dcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.425516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6989d-32c7-4640-bc7a-f68c3812dcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.448059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-69b6989d-32c7-4640-bc7a-f68c3812dcb7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:57.449418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-69b6989d-32c7-4640-bc7a-f68c3812dcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:57.518703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1062/2120] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:57.526446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf0ce-72a6-4eff-9d6a-5118de5fb650 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.526731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf0ce-72a6-4eff-9d6a-5118de5fb650 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.527005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf0ce-72a6-4eff-9d6a-5118de5fb650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.548793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-89cdf0ce-72a6-4eff-9d6a-5118de5fb650 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:57.550259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-89cdf0ce-72a6-4eff-9d6a-5118de5fb650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:57.565047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1059/2121] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:57.566432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6989d-32c7-4640-bc7a-f68c3812dcb7 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:57.580359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6989d-32c7-4640-bc7a-f68c3812dcb7 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:57.585217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:57.585825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.586267 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.586713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.587523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:29:57.587987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:57.589785 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:57.605885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1063/2122] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:29:57.610188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdf0ce-72a6-4eff-9d6a-5118de5fb650 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:57.621237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdf0ce-72a6-4eff-9d6a-5118de5fb650 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:57.628472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdba47-fc64-4190-831d-577111f2e0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:57.629261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdba47-fc64-4190-831d-577111f2e0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.629533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdba47-fc64-4190-831d-577111f2e0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.630138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdba47-fc64-4190-831d-577111f2e0b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.630799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdba47-fc64-4190-831d-577111f2e0b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:57.633127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdba47-fc64-4190-831d-577111f2e0b2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:57.674720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:57.680231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1060/2123] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:57 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:57.687029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e1652-5aa3-4301-9eb1-c32256480af5 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:57.767586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbd886-03e7-4177-9f1b-3e1d9f607a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:57.768403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbd886-03e7-4177-9f1b-3e1d9f607a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.768691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbd886-03e7-4177-9f1b-3e1d9f607a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.768967 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbd886-03e7-4177-9f1b-3e1d9f607a81 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.769723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbd886-03e7-4177-9f1b-3e1d9f607a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:57.772126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbd886-03e7-4177-9f1b-3e1d9f607a81 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:57.815661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1064/2124] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:57 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:57.817157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdba47-fc64-4190-831d-577111f2e0b2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:57.839894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdba47-fc64-4190-831d-577111f2e0b2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:57.846060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5be98-85f3-4a80-bacb-6cd514471ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:57.846799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5be98-85f3-4a80-bacb-6cd514471ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.847182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5be98-85f3-4a80-bacb-6cd514471ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.847304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5be98-85f3-4a80-bacb-6cd514471ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.848642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5be98-85f3-4a80-bacb-6cd514471ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:57.850396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5be98-85f3-4a80-bacb-6cd514471ac6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:57.867632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1061/2125] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:57.869048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbd886-03e7-4177-9f1b-3e1d9f607a81 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:57.883349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbd886-03e7-4177-9f1b-3e1d9f607a81 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:57.897325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbde9dd-8e7d-44c7-80d0-31cc799956b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:57.898102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbde9dd-8e7d-44c7-80d0-31cc799956b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.898486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbde9dd-8e7d-44c7-80d0-31cc799956b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.898860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbde9dd-8e7d-44c7-80d0-31cc799956b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.899605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbde9dd-8e7d-44c7-80d0-31cc799956b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:57.902999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbde9dd-8e7d-44c7-80d0-31cc799956b3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:57.913139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:57.922259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1065/2126] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:57.927603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5be98-85f3-4a80-bacb-6cd514471ac6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:57.944166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5be98-85f3-4a80-bacb-6cd514471ac6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:57.952227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b59435-1e01-4c51-a8ae-20f354b58eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:57.953642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b59435-1e01-4c51-a8ae-20f354b58eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:57.953928 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b59435-1e01-4c51-a8ae-20f354b58eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:57.954385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b59435-1e01-4c51-a8ae-20f354b58eb5 admin admin] PATH_INFO: `/v3/projects/2e9d2b82f29843c696bcf8f703edfe1d/users/7a7abfc2622a4d10a6ce38f758af4b65/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:57.983534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b59435-1e01-4c51-a8ae-20f354b58eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9d2b82f29843c696bcf8f703edfe1d, user_id=7a7abfc2622a4d10a6ce38f758af4b65, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:57.986729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b59435-1e01-4c51-a8ae-20f354b58eb5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:58.004767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1062/2127] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:57 2022] PUT /identity/v3/projects/2e9d2b82f29843c696bcf8f703edfe1d/users/7a7abfc2622a4d10a6ce38f758af4b65/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:29:58.017841 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edfde6-2aef-440f-ac4c-03c837a0b7dd None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:58.018127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edfde6-2aef-440f-ac4c-03c837a0b7dd None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:58.018398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39edfde6-2aef-440f-ac4c-03c837a0b7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:58.090124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-39edfde6-2aef-440f-ac4c-03c837a0b7dd None None] MFA Rules not processed for user `7a7abfc2622a4d10a6ce38f758af4b65`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:58.091377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-39edfde6-2aef-440f-ac4c-03c837a0b7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:58.215540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1066/2128] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:58.220735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b59435-1e01-4c51-a8ae-20f354b58eb5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:58.314307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:58.357082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b59435-1e01-4c51-a8ae-20f354b58eb5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:58.364336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00b3f2-03c5-4e1e-a682-ced3988aa2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:58.365092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b3f2-03c5-4e1e-a682-ced3988aa2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:58.365623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b3f2-03c5-4e1e-a682-ced3988aa2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:58.365956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b3f2-03c5-4e1e-a682-ced3988aa2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:58.367289 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00b3f2-03c5-4e1e-a682-ced3988aa2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:58.369868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00b3f2-03c5-4e1e-a682-ced3988aa2d2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:58.413700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1063/2129] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:29:58 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:29:58.414881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edfde6-2aef-440f-ac4c-03c837a0b7dd None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:58.427969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39edfde6-2aef-440f-ac4c-03c837a0b7dd None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:58.511703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39efc4-df8d-4956-8067-4dfb64d76456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:58.512822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39efc4-df8d-4956-8067-4dfb64d76456 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:58.513256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39efc4-df8d-4956-8067-4dfb64d76456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:58.513642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39efc4-df8d-4956-8067-4dfb64d76456 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:58.514427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39efc4-df8d-4956-8067-4dfb64d76456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:58.516722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39efc4-df8d-4956-8067-4dfb64d76456 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:58.543799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1067/2130] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:29:58 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:29:58.548710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00b3f2-03c5-4e1e-a682-ced3988aa2d2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:58.561718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00b3f2-03c5-4e1e-a682-ced3988aa2d2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:58.567420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72522c66-dd9f-4e2f-8363-059de0c4d570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:58.568439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72522c66-dd9f-4e2f-8363-059de0c4d570 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:58.568793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72522c66-dd9f-4e2f-8363-059de0c4d570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:58.569137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72522c66-dd9f-4e2f-8363-059de0c4d570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:58.569866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72522c66-dd9f-4e2f-8363-059de0c4d570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:58.571543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72522c66-dd9f-4e2f-8363-059de0c4d570 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:58.589584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1064/2131] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:58.593619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39efc4-df8d-4956-8067-4dfb64d76456 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:58.604599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39efc4-df8d-4956-8067-4dfb64d76456 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:58.610444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e31c9a-f26f-4559-8112-22f938bbcb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:58.611185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e31c9a-f26f-4559-8112-22f938bbcb4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:58.611543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e31c9a-f26f-4559-8112-22f938bbcb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:58.611902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e31c9a-f26f-4559-8112-22f938bbcb4f admin admin] PATH_INFO: `/v3/projects/d1d6163c7420448680b769649db17730/users/a8a75261db4243f5bf7c0ccb083b59d4/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:58.628031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e31c9a-f26f-4559-8112-22f938bbcb4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d6163c7420448680b769649db17730, user_id=a8a75261db4243f5bf7c0ccb083b59d4, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:58.631863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e31c9a-f26f-4559-8112-22f938bbcb4f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:58.646380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1068/2132] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:58 2022] PUT /identity/v3/projects/d1d6163c7420448680b769649db17730/users/a8a75261db4243f5bf7c0ccb083b59d4/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:58.651370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72522c66-dd9f-4e2f-8363-059de0c4d570 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:58.663242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72522c66-dd9f-4e2f-8363-059de0c4d570 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:58.727546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3709ca23-fce8-4e8e-920f-87b1aaba2a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:58.728505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ca23-fce8-4e8e-920f-87b1aaba2a59 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:58.729206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ca23-fce8-4e8e-920f-87b1aaba2a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:58.729808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3709ca23-fce8-4e8e-920f-87b1aaba2a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:58.730565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709ca23-fce8-4e8e-920f-87b1aaba2a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:58.732740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3709ca23-fce8-4e8e-920f-87b1aaba2a59 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:58.744586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1065/2133] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:29:58.748901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e31c9a-f26f-4559-8112-22f938bbcb4f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:58.760404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e31c9a-f26f-4559-8112-22f938bbcb4f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:58.765068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:58.765766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:58.766791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:58.767073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] PATH_INFO: `/v3/projects/d1d6163c7420448680b769649db17730/users/a8a75261db4243f5bf7c0ccb083b59d4/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:58.769660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d6163c7420448680b769649db17730, user_id=a8a75261db4243f5bf7c0ccb083b59d4, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:58.773154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:58.784057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1069/2134] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:29:58 2022] PUT /identity/v3/projects/d1d6163c7420448680b769649db17730/users/a8a75261db4243f5bf7c0ccb083b59d4/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:58.794519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af3e5f-a9ab-4de9-8996-968c1ad5e010 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:58.794806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af3e5f-a9ab-4de9-8996-968c1ad5e010 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:58.795083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af3e5f-a9ab-4de9-8996-968c1ad5e010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:58.825609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-77af3e5f-a9ab-4de9-8996-968c1ad5e010 None None] MFA Rules not processed for user `a8a75261db4243f5bf7c0ccb083b59d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:58.827078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-77af3e5f-a9ab-4de9-8996-968c1ad5e010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:58.916175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1066/2135] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:29:58.925183 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:58.928696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:58.985173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:58.999501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe0fb1-1446-4ea4-abe0-7fecac9b5aa4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:59.003408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f7e50-a8f1-4abd-b38e-e4f0de9158ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:59.004060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f7e50-a8f1-4abd-b38e-e4f0de9158ec service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:59.004361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f7e50-a8f1-4abd-b38e-e4f0de9158ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:59.004630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f7e50-a8f1-4abd-b38e-e4f0de9158ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:59.014376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f7e50-a8f1-4abd-b38e-e4f0de9158ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:59.015840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f7e50-a8f1-4abd-b38e-e4f0de9158ec service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:59.026710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1070/2136] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:59.110413 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af3e5f-a9ab-4de9-8996-968c1ad5e010 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:59.132222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af3e5f-a9ab-4de9-8996-968c1ad5e010 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:59.137293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecde88-6ddb-46ac-b62c-cacd05e6c207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:59.138093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecde88-6ddb-46ac-b62c-cacd05e6c207 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:59.138491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecde88-6ddb-46ac-b62c-cacd05e6c207 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:59.139030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecde88-6ddb-46ac-b62c-cacd05e6c207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:59.176137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:29:59.216578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecde88-6ddb-46ac-b62c-cacd05e6c207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:59.218006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecde88-6ddb-46ac-b62c-cacd05e6c207 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:59.227165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1067/2137] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:29:59.555781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f7e50-a8f1-4abd-b38e-e4f0de9158ec service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:59.626585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f7e50-a8f1-4abd-b38e-e4f0de9158ec service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:59.632344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:59.632980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:59.633262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:59.633531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] PATH_INFO: `/v3/users/7a7abfc2622a4d10a6ce38f758af4b65` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:59.634688 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a7abfc2622a4d10a6ce38f758af4b65)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:29:59.636656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:29:59.724843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a7abfc2622a4d10a6ce38f758af4b65'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:59.752328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a7abfc2622a4d10a6ce38f758af4b65'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:59.753413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a7abfc2622a4d10a6ce38f758af4b65'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:59.765577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a7abfc2622a4d10a6ce38f758af4b65'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:59.783573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a7abfc2622a4d10a6ce38f758af4b65'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:29:59.795651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1071/2138] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:29:59 2022] DELETE /identity/v3/users/7a7abfc2622a4d10a6ce38f758af4b65 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:29:59.801462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5164d8-ccda-418f-bc56-7b29a3fccf9f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:59.802457 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5164d8-ccda-418f-bc56-7b29a3fccf9f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:59.802941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5164d8-ccda-418f-bc56-7b29a3fccf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:29:59.827892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ff5164d8-ccda-418f-bc56-7b29a3fccf9f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:29:59.829795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ff5164d8-ccda-418f-bc56-7b29a3fccf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:59.905952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1068/2139] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:29:59.916944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:29:59.918410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:29:59.984203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6355b3-3faa-4f89-9851-ef075d8b5c31 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:29:59.987990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceab3ed-f401-4dbf-a919-abd322cdc003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:29:59.989247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab3ed-f401-4dbf-a919-abd322cdc003 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:29:59.989809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab3ed-f401-4dbf-a919-abd322cdc003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:29:59.989962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab3ed-f401-4dbf-a919-abd322cdc003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:00.001273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceab3ed-f401-4dbf-a919-abd322cdc003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:00.002627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceab3ed-f401-4dbf-a919-abd322cdc003 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:00.018034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1072/2140] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:00.364882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5164d8-ccda-418f-bc56-7b29a3fccf9f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:00.367151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ff5164d8-ccda-418f-bc56-7b29a3fccf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:00.383446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5164d8-ccda-418f-bc56-7b29a3fccf9f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:00.388526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:00.389250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:00.389760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:00.390322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] PATH_INFO: `/v3/projects/2e9d2b82f29843c696bcf8f703edfe1d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:00.391590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e9d2b82f29843c696bcf8f703edfe1d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:00.393761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:00.468776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e9d2b82f29843c696bcf8f703edfe1d', 'request_id': 'req-30e061d4-37c2-46b9-82f1-92c90cd9871f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.478109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e9d2b82f29843c696bcf8f703edfe1d', 'request_id': 'req-30e061d4-37c2-46b9-82f1-92c90cd9871f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.491533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e9d2b82f29843c696bcf8f703edfe1d', 'request_id': 'req-30e061d4-37c2-46b9-82f1-92c90cd9871f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.504530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] The token cache is being invalidate because project 2e9d2b82f29843c696bcf8f703edfe1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:30:00.504971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.505637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.507178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1069/2141] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:30:00 2022] DELETE /identity/v3/projects/2e9d2b82f29843c696bcf8f703edfe1d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:00.511190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceab3ed-f401-4dbf-a919-abd322cdc003 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:00.523354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceab3ed-f401-4dbf-a919-abd322cdc003 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:00.527085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:00.590171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:00.591056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:00.591056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:00.591347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] PATH_INFO: `/v3/users/a8a75261db4243f5bf7c0ccb083b59d4` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:00.592631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a75261db4243f5bf7c0ccb083b59d4)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:00.595040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:00.649574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a75261db4243f5bf7c0ccb083b59d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.664938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a75261db4243f5bf7c0ccb083b59d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.665666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a75261db4243f5bf7c0ccb083b59d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.679693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a75261db4243f5bf7c0ccb083b59d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.689428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a47d2788-252a-406a-a8e4-41fee5f62967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a75261db4243f5bf7c0ccb083b59d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:00.697224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1073/2142] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:30:00 2022] DELETE /identity/v3/users/a8a75261db4243f5bf7c0ccb083b59d4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:01.074673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:01.076503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:01.096803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e061d4-37c2-46b9-82f1-92c90cd9871f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:01.156557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:01.169716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:01.170568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.170939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.171409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] PATH_INFO: `/v3/projects/d1d6163c7420448680b769649db17730` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.172750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d6163c7420448680b769649db17730)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:01.174743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:01.228088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d6163c7420448680b769649db17730', 'request_id': 'req-4ec63803-4558-4232-b2b2-3cde25d91342'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:01.243149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d6163c7420448680b769649db17730', 'request_id': 'req-4ec63803-4558-4232-b2b2-3cde25d91342'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:01.257407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d6163c7420448680b769649db17730', 'request_id': 'req-4ec63803-4558-4232-b2b2-3cde25d91342'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:01.277606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] The token cache is being invalidate because project d1d6163c7420448680b769649db17730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:30:01.277892 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:01.278850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4ec63803-4558-4232-b2b2-3cde25d91342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:01.285461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1070/2143] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:30:01 2022] DELETE /identity/v3/projects/d1d6163c7420448680b769649db17730 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:01.288531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d641843-1a32-4af5-ba12-5175d1f48ca5 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.289081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d641843-1a32-4af5-ba12-5175d1f48ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.289536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d641843-1a32-4af5-ba12-5175d1f48ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.311651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4d641843-1a32-4af5-ba12-5175d1f48ca5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:01.312952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4d641843-1a32-4af5-ba12-5175d1f48ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:01.395736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1074/2144] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:01.403172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a4233-6ffb-47e9-a066-4d07a5aee611 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.403969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a4233-6ffb-47e9-a066-4d07a5aee611 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.404486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a4233-6ffb-47e9-a066-4d07a5aee611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.424833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c97a4233-6ffb-47e9-a066-4d07a5aee611 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:01.426836 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c97a4233-6ffb-47e9-a066-4d07a5aee611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:01.435201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1071/2145] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:30:01.439508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d641843-1a32-4af5-ba12-5175d1f48ca5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:01.449105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d641843-1a32-4af5-ba12-5175d1f48ca5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:01.453568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:01.454147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.454416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.454677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.455313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:30:01.455611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:01.457241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:01.468445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1075/2146] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:30:01.473385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a4233-6ffb-47e9-a066-4d07a5aee611 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:01.482530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a4233-6ffb-47e9-a066-4d07a5aee611 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:01.487187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae1707-b3bf-4c09-8d14-489230f55be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:01.487775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae1707-b3bf-4c09-8d14-489230f55be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.488229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae1707-b3bf-4c09-8d14-489230f55be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.489149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae1707-b3bf-4c09-8d14-489230f55be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.489784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae1707-b3bf-4c09-8d14-489230f55be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:01.491761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbae1707-b3bf-4c09-8d14-489230f55be1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:01.531131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1072/2147] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:30:01 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:30:01.532536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:01.545231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd9012-8579-4b3a-bda6-2ade19810ff1 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:01.565513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:01.617374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68013957-7580-465b-be0e-d3d77f34c961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:01.618039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68013957-7580-465b-be0e-d3d77f34c961 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.618311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68013957-7580-465b-be0e-d3d77f34c961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.618590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68013957-7580-465b-be0e-d3d77f34c961 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.619231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68013957-7580-465b-be0e-d3d77f34c961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:01.621237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68013957-7580-465b-be0e-d3d77f34c961 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:01.646842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1076/2148] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:30:01 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:30:01.651027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae1707-b3bf-4c09-8d14-489230f55be1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:01.661448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae1707-b3bf-4c09-8d14-489230f55be1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:01.666026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12a4a5-5790-4d6e-afa5-5a7e8a7134d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:01.666621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12a4a5-5790-4d6e-afa5-5a7e8a7134d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.666898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12a4a5-5790-4d6e-afa5-5a7e8a7134d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.667168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12a4a5-5790-4d6e-afa5-5a7e8a7134d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.667806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12a4a5-5790-4d6e-afa5-5a7e8a7134d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:01.669428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12a4a5-5790-4d6e-afa5-5a7e8a7134d3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:01.679158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1073/2149] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:30:01.683403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68013957-7580-465b-be0e-d3d77f34c961 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:01.694058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68013957-7580-465b-be0e-d3d77f34c961 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:01.699734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28d553-eb6c-45d2-8d12-d99cd62e01ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:01.700474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d553-eb6c-45d2-8d12-d99cd62e01ac admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.700749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d553-eb6c-45d2-8d12-d99cd62e01ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.701013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28d553-eb6c-45d2-8d12-d99cd62e01ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.701647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28d553-eb6c-45d2-8d12-d99cd62e01ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:01.703249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28d553-eb6c-45d2-8d12-d99cd62e01ac admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:01.714388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1077/2150] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:30:01.718686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12a4a5-5790-4d6e-afa5-5a7e8a7134d3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:01.728592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12a4a5-5790-4d6e-afa5-5a7e8a7134d3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:01.735498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:01.736981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.737540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.738619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] PATH_INFO: `/v3/projects/7e7adf1b3fc044eeaf5da0173d4276a7/users/b995a81b93264d99bf476c02eb319926/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.763117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7adf1b3fc044eeaf5da0173d4276a7, user_id=b995a81b93264d99bf476c02eb319926, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:01.766410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:01.787522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1074/2151] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:30:01 2022] PUT /identity/v3/projects/7e7adf1b3fc044eeaf5da0173d4276a7/users/b995a81b93264d99bf476c02eb319926/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:01.802797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd67d1-40df-428f-a52f-00dc82f55271 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:01.803206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd67d1-40df-428f-a52f-00dc82f55271 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:01.803735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd67d1-40df-428f-a52f-00dc82f55271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:01.846405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-49cd67d1-40df-428f-a52f-00dc82f55271 None None] MFA Rules not processed for user `b995a81b93264d99bf476c02eb319926`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:01.848275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-49cd67d1-40df-428f-a52f-00dc82f55271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:01.958491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1078/2152] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:01.969641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:01.971326 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:02.029070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18410c-8e22-42bd-87e4-a69fe17467b6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:02.032730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37746b47-1847-4602-a716-01131f3ce38b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:02.033333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37746b47-1847-4602-a716-01131f3ce38b service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:02.033613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37746b47-1847-4602-a716-01131f3ce38b service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:02.033889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37746b47-1847-4602-a716-01131f3ce38b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:02.043220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37746b47-1847-4602-a716-01131f3ce38b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:02.044711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37746b47-1847-4602-a716-01131f3ce38b service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:02.053857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1075/2153] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:02.080674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd67d1-40df-428f-a52f-00dc82f55271 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:02.148933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd67d1-40df-428f-a52f-00dc82f55271 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:02.153458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:02.154082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:02.154358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:02.154620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] PATH_INFO: `/v3/users/b995a81b93264d99bf476c02eb319926` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:02.155758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b995a81b93264d99bf476c02eb319926)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:02.157707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:02.180231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:02.216168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b995a81b93264d99bf476c02eb319926'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:02.232881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b995a81b93264d99bf476c02eb319926'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:02.233627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b995a81b93264d99bf476c02eb319926'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:02.243704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b995a81b93264d99bf476c02eb319926'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:02.256294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b995a81b93264d99bf476c02eb319926'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:02.262907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1079/2154] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:30:02 2022] DELETE /identity/v3/users/b995a81b93264d99bf476c02eb319926 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:02.267508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2f84d-cf1b-4ed4-89bf-5edad54a6f7c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:02.267799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2f84d-cf1b-4ed4-89bf-5edad54a6f7c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:02.268142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2f84d-cf1b-4ed4-89bf-5edad54a6f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:02.288931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-51c2f84d-cf1b-4ed4-89bf-5edad54a6f7c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:02.290118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-51c2f84d-cf1b-4ed4-89bf-5edad54a6f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:02.306799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:02.363859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1076/2155] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:02.377133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:02.396910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:02.482172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a960a-a303-4141-8191-64c8444b00c3 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:02.485901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f4f0e-3870-48bc-8574-294268bf652e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:02.486529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f4f0e-3870-48bc-8574-294268bf652e service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:02.486823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f4f0e-3870-48bc-8574-294268bf652e service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:02.487113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f4f0e-3870-48bc-8574-294268bf652e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:02.497916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f4f0e-3870-48bc-8574-294268bf652e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:02.499562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f4f0e-3870-48bc-8574-294268bf652e service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:02.511170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1080/2156] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:03.011308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2f84d-cf1b-4ed4-89bf-5edad54a6f7c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:03.014210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-51c2f84d-cf1b-4ed4-89bf-5edad54a6f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:03.041405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2f84d-cf1b-4ed4-89bf-5edad54a6f7c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:03.063834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:03.066991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.067469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.069523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] PATH_INFO: `/v3/projects/7e7adf1b3fc044eeaf5da0173d4276a7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.073621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e7adf1b3fc044eeaf5da0173d4276a7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:03.077798 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:03.152733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7adf1b3fc044eeaf5da0173d4276a7', 'request_id': 'req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:03.164476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7adf1b3fc044eeaf5da0173d4276a7', 'request_id': 'req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:03.180365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7adf1b3fc044eeaf5da0173d4276a7', 'request_id': 'req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:03.196570 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] The token cache is being invalidate because project 7e7adf1b3fc044eeaf5da0173d4276a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:30:03.196887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:03.200430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8ec0c6cb-de5a-4553-96a9-e73b3f29e957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:03.202886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1077/2157] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:30:03 2022] DELETE /identity/v3/projects/7e7adf1b3fc044eeaf5da0173d4276a7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:30:03.207656 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f7c35-ea05-45da-9387-0a0b95fd7b6c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.208544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f7c35-ea05-45da-9387-0a0b95fd7b6c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.208814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f7c35-ea05-45da-9387-0a0b95fd7b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.234651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-875f7c35-ea05-45da-9387-0a0b95fd7b6c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:03.235846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-875f7c35-ea05-45da-9387-0a0b95fd7b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:03.324587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1081/2158] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:03.332644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0704-376b-450d-b642-7f50130d3feb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.333181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0704-376b-450d-b642-7f50130d3feb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.333610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b0704-376b-450d-b642-7f50130d3feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.354612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-209b0704-376b-450d-b642-7f50130d3feb None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:03.355979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-209b0704-376b-450d-b642-7f50130d3feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:03.367200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1078/2159] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:30:03.371828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f7c35-ea05-45da-9387-0a0b95fd7b6c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:03.386837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f7c35-ea05-45da-9387-0a0b95fd7b6c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:03.394428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:03.395062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.395347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.395614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.396304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:30:03.396630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:03.398831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:03.416775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1082/2160] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:30:03.420834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b0704-376b-450d-b642-7f50130d3feb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:03.434903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b0704-376b-450d-b642-7f50130d3feb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:03.452940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6d3f2-2e5d-4634-ab94-a80b08503694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:03.453838 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d3f2-2e5d-4634-ab94-a80b08503694 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.454213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d3f2-2e5d-4634-ab94-a80b08503694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.454579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d3f2-2e5d-4634-ab94-a80b08503694 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.455302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6d3f2-2e5d-4634-ab94-a80b08503694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:03.457342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6d3f2-2e5d-4634-ab94-a80b08503694 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:03.509823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1079/2161] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:30:03 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:30:03.514392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:03.527327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3a264-6dd1-4b31-bb36-d6452d647c2a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:03.561781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:03.596318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1757b-862c-4c03-8f2b-9db9b51200b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:03.597239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1757b-862c-4c03-8f2b-9db9b51200b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.598260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1757b-862c-4c03-8f2b-9db9b51200b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.598704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1757b-862c-4c03-8f2b-9db9b51200b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.599513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1757b-862c-4c03-8f2b-9db9b51200b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:03.601653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1757b-862c-4c03-8f2b-9db9b51200b9 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:03.634175 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1083/2162] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:30:03 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:30:03.635368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6d3f2-2e5d-4634-ab94-a80b08503694 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:03.647181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6d3f2-2e5d-4634-ab94-a80b08503694 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:03.658287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818c517-251f-4779-9bf1-3031acca05c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:03.659127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818c517-251f-4779-9bf1-3031acca05c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.659542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818c517-251f-4779-9bf1-3031acca05c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.660571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818c517-251f-4779-9bf1-3031acca05c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.661918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818c517-251f-4779-9bf1-3031acca05c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:03.664233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818c517-251f-4779-9bf1-3031acca05c6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:03.678229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1080/2163] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:30:03.683368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1757b-862c-4c03-8f2b-9db9b51200b9 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:03.701898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1757b-862c-4c03-8f2b-9db9b51200b9 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:03.715460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351186e9-e3a0-437b-9d6c-970d432cf2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:03.716272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351186e9-e3a0-437b-9d6c-970d432cf2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.716557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351186e9-e3a0-437b-9d6c-970d432cf2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.716828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351186e9-e3a0-437b-9d6c-970d432cf2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.717490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351186e9-e3a0-437b-9d6c-970d432cf2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:03.719152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351186e9-e3a0-437b-9d6c-970d432cf2e9 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:03.731931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1084/2164] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:30:03.750454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818c517-251f-4779-9bf1-3031acca05c6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:03.764163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818c517-251f-4779-9bf1-3031acca05c6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:03.771154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:03.771918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.772299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.772577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] PATH_INFO: `/v3/projects/3c8f0998170342a98ce61483a0ee2e50/users/9ec06f6d349842edbaaba4e6c3ff6885/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.794726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c8f0998170342a98ce61483a0ee2e50, user_id=9ec06f6d349842edbaaba4e6c3ff6885, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:03.797607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:03.815637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1081/2165] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:30:03 2022] PUT /identity/v3/projects/3c8f0998170342a98ce61483a0ee2e50/users/9ec06f6d349842edbaaba4e6c3ff6885/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:03.826706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a54cd-8d56-47ad-b392-67ec13792409 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:03.827055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a54cd-8d56-47ad-b392-67ec13792409 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:03.827262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a54cd-8d56-47ad-b392-67ec13792409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:03.876299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-892a54cd-8d56-47ad-b392-67ec13792409 None None] MFA Rules not processed for user `9ec06f6d349842edbaaba4e6c3ff6885`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:03.877971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-892a54cd-8d56-47ad-b392-67ec13792409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:03.961260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1085/2166] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:03.971900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:03.974009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:04.025628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:04.046989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38c71a-3024-4d2b-b0d7-eb3916f5ccd8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:04.051695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59abb49-245a-4d30-abb9-4c1876c327d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:04.052725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59abb49-245a-4d30-abb9-4c1876c327d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:04.053062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59abb49-245a-4d30-abb9-4c1876c327d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:04.053507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59abb49-245a-4d30-abb9-4c1876c327d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:04.069048 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59abb49-245a-4d30-abb9-4c1876c327d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:04.070408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59abb49-245a-4d30-abb9-4c1876c327d0 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:04.080412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1082/2167] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:09.986934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeac6-cfaa-466e-bcde-ab90e49473fa None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:09.987427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeac6-cfaa-466e-bcde-ab90e49473fa None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:09.987427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aeac6-cfaa-466e-bcde-ab90e49473fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:10.012256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-216aeac6-cfaa-466e-bcde-ab90e49473fa None None] MFA Rules not processed for user `ea2b7595ca6d446c84e95a05a701e9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:10.013198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-216aeac6-cfaa-466e-bcde-ab90e49473fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:10.132337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1086/2168] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:13.293288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59abb49-245a-4d30-abb9-4c1876c327d0 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:13.410685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59abb49-245a-4d30-abb9-4c1876c327d0 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:13.428254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:13.430172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:13.430838 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:13.431764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] PATH_INFO: `/v3/users/9ec06f6d349842edbaaba4e6c3ff6885` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:13.433595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec06f6d349842edbaaba4e6c3ff6885)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:13.437202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:13.554734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec06f6d349842edbaaba4e6c3ff6885'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:13.574973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec06f6d349842edbaaba4e6c3ff6885'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:13.576700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec06f6d349842edbaaba4e6c3ff6885'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:13.621697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec06f6d349842edbaaba4e6c3ff6885'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:13.622493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec06f6d349842edbaaba4e6c3ff6885'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:13.658170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1083/2169] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:30:13 2022] DELETE /identity/v3/users/9ec06f6d349842edbaaba4e6c3ff6885 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:13.659437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fec4c5-a4b5-43a9-8681-b9a35eae8eb5 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:13.660214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fec4c5-a4b5-43a9-8681-b9a35eae8eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:13.660214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fec4c5-a4b5-43a9-8681-b9a35eae8eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:13.825225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d7fec4c5-a4b5-43a9-8681-b9a35eae8eb5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:13.837087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d7fec4c5-a4b5-43a9-8681-b9a35eae8eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:14.122951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:14.193845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1087/2170] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:14.213432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:14.215193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:14.554517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240ac74-7f0e-4335-9707-2359261c4792 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:14.577941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52678e48-5c04-413b-9a15-f52f34dd2d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:14.578709 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52678e48-5c04-413b-9a15-f52f34dd2d7c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:14.579671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52678e48-5c04-413b-9a15-f52f34dd2d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:14.579960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52678e48-5c04-413b-9a15-f52f34dd2d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:14.603455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52678e48-5c04-413b-9a15-f52f34dd2d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:14.605277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52678e48-5c04-413b-9a15-f52f34dd2d7c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:14.636180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1084/2171] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:15.693997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fec4c5-a4b5-43a9-8681-b9a35eae8eb5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:15.699517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d7fec4c5-a4b5-43a9-8681-b9a35eae8eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:15.715997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fec4c5-a4b5-43a9-8681-b9a35eae8eb5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:15.722232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:15.723678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:15.724185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:15.724561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] PATH_INFO: `/v3/projects/3c8f0998170342a98ce61483a0ee2e50` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:15.726013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c8f0998170342a98ce61483a0ee2e50)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:15.728043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:15.813030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c8f0998170342a98ce61483a0ee2e50', 'request_id': 'req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:15.823588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c8f0998170342a98ce61483a0ee2e50', 'request_id': 'req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:15.830435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c8f0998170342a98ce61483a0ee2e50', 'request_id': 'req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:15.864190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] The token cache is being invalidate because project 3c8f0998170342a98ce61483a0ee2e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:30:15.865037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:15.866527 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4843bf97-93f9-4a6b-8dfc-f1f4d38be320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:15.868556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1088/2172] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:30:15 2022] DELETE /identity/v3/projects/3c8f0998170342a98ce61483a0ee2e50 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:15.873556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1939a3c-e97e-487e-ae59-575e5e79e216 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:15.874105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1939a3c-e97e-487e-ae59-575e5e79e216 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:15.874206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1939a3c-e97e-487e-ae59-575e5e79e216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:15.896973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:15.907989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e1939a3c-e97e-487e-ae59-575e5e79e216 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:15.911531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e1939a3c-e97e-487e-ae59-575e5e79e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:16.001879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1085/2173] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:16.010681 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a60772-47ee-4b1b-a6db-0e7b7600b3e5 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.011742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a60772-47ee-4b1b-a6db-0e7b7600b3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.012595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a60772-47ee-4b1b-a6db-0e7b7600b3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.045920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e7a60772-47ee-4b1b-a6db-0e7b7600b3e5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:16.047161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e7a60772-47ee-4b1b-a6db-0e7b7600b3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:16.070807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1089/2174] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:16.072458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1939a3c-e97e-487e-ae59-575e5e79e216 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:16.085206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1939a3c-e97e-487e-ae59-575e5e79e216 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:16.105657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:16.106503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.106889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.107263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.108181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:30:16.108837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:16.110728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:16.126782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1086/2175] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:30:16.131117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a60772-47ee-4b1b-a6db-0e7b7600b3e5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:16.147939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a60772-47ee-4b1b-a6db-0e7b7600b3e5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:16.159019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7001d5-ce2c-43d0-b3bf-16b71c32bfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:16.160394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7001d5-ce2c-43d0-b3bf-16b71c32bfc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.161129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7001d5-ce2c-43d0-b3bf-16b71c32bfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.161129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7001d5-ce2c-43d0-b3bf-16b71c32bfc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.161540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7001d5-ce2c-43d0-b3bf-16b71c32bfc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:16.170269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7001d5-ce2c-43d0-b3bf-16b71c32bfc6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:16.219588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1090/2176] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:30:16 2022] POST /identity/v3/projects => generated 381 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:30:16.221689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:16.239978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a8021-b384-469c-a408-33465fc48ffe admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:16.341258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371af69-a186-4cfc-8836-4dd12f3c94fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:16.342539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371af69-a186-4cfc-8836-4dd12f3c94fb admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.343622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371af69-a186-4cfc-8836-4dd12f3c94fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.344057 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371af69-a186-4cfc-8836-4dd12f3c94fb admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.344898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371af69-a186-4cfc-8836-4dd12f3c94fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:16.351549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371af69-a186-4cfc-8836-4dd12f3c94fb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:16.390287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1087/2177] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:30:16 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:30:16.394481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7001d5-ce2c-43d0-b3bf-16b71c32bfc6 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:16.413348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7001d5-ce2c-43d0-b3bf-16b71c32bfc6 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:16.420212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10ca64-c18a-443a-bf09-b61017c3223f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:16.421896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10ca64-c18a-443a-bf09-b61017c3223f admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.422948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10ca64-c18a-443a-bf09-b61017c3223f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.423937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10ca64-c18a-443a-bf09-b61017c3223f admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.425538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10ca64-c18a-443a-bf09-b61017c3223f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:16.428289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10ca64-c18a-443a-bf09-b61017c3223f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:16.447814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1091/2178] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:30:16.453889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371af69-a186-4cfc-8836-4dd12f3c94fb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:16.471496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371af69-a186-4cfc-8836-4dd12f3c94fb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:16.481556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4efe0-9511-472f-83f5-957d97497289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:16.482199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4efe0-9511-472f-83f5-957d97497289 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.482481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4efe0-9511-472f-83f5-957d97497289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.482866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4efe0-9511-472f-83f5-957d97497289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.483529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4efe0-9511-472f-83f5-957d97497289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:16.485240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4efe0-9511-472f-83f5-957d97497289 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:16.508331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1088/2179] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:30:16.524519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10ca64-c18a-443a-bf09-b61017c3223f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:16.536719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10ca64-c18a-443a-bf09-b61017c3223f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:16.558523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:16.559410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.559693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.559965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] PATH_INFO: `/v3/projects/92e71ba59c6d42dabeef49f62cebac1d/users/992a34be52b7491da0b68cbe17f25b39/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.581367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e71ba59c6d42dabeef49f62cebac1d, user_id=992a34be52b7491da0b68cbe17f25b39, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:16.585189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:16.599347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1092/2180] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:30:16 2022] PUT /identity/v3/projects/92e71ba59c6d42dabeef49f62cebac1d/users/992a34be52b7491da0b68cbe17f25b39/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:16.604470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f862756-36e4-436d-9cda-ebe2b7bffe0e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.604759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f862756-36e4-436d-9cda-ebe2b7bffe0e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.605037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f862756-36e4-436d-9cda-ebe2b7bffe0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.625258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5f862756-36e4-436d-9cda-ebe2b7bffe0e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:16.626008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5f862756-36e4-436d-9cda-ebe2b7bffe0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:16.693366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1089/2181] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:16.704090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:16.705949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:16.782864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67daeb7-6180-474b-9f56-7f923384e434 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:16.787608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e52bb-262a-4358-b394-2d92c34a9e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:16.788280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e52bb-262a-4358-b394-2d92c34a9e86 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:16.788568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e52bb-262a-4358-b394-2d92c34a9e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:16.788851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e52bb-262a-4358-b394-2d92c34a9e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:16.799014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e52bb-262a-4358-b394-2d92c34a9e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:16.800686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e52bb-262a-4358-b394-2d92c34a9e86 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:16.811977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1093/2182] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:17.600576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cfc92-5a2e-44d4-9f98-6ed0041a0fd9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:17.601037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cfc92-5a2e-44d4-9f98-6ed0041a0fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:17.601155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cfc92-5a2e-44d4-9f98-6ed0041a0fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:17.643734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:17.652785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ed9cfc92-5a2e-44d4-9f98-6ed0041a0fd9 None None] MFA Rules not processed for user `992a34be52b7491da0b68cbe17f25b39`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:17.656272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ed9cfc92-5a2e-44d4-9f98-6ed0041a0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:17.675128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09765b93-a4f1-44f3-a986-0d6cdf167cc8 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:17.675444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09765b93-a4f1-44f3-a986-0d6cdf167cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:17.675699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09765b93-a4f1-44f3-a986-0d6cdf167cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:17.694472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:17.705461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-09765b93-a4f1-44f3-a986-0d6cdf167cc8 None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:17.709149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-09765b93-a4f1-44f3-a986-0d6cdf167cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:17.773241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1090/2183] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:17.782354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cfc92-5a2e-44d4-9f98-6ed0041a0fd9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:17.783525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ed9cfc92-5a2e-44d4-9f98-6ed0041a0fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:17.824025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1094/2184] 10.222.0.43 () {58 vars in 967 bytes} [Thu Aug 11 16:30:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:30:17.845494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09765b93-a4f1-44f3-a986-0d6cdf167cc8 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:17.848673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-09765b93-a4f1-44f3-a986-0d6cdf167cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:17.872047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cfc92-5a2e-44d4-9f98-6ed0041a0fd9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:17.877280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8918c-0871-4900-825f-d33ee8f242a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:17.878123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8918c-0871-4900-825f-d33ee8f242a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:17.879263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8918c-0871-4900-825f-d33ee8f242a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:17.879840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8918c-0871-4900-825f-d33ee8f242a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:17.893666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8918c-0871-4900-825f-d33ee8f242a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:17.896314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8918c-0871-4900-825f-d33ee8f242a2 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:17.907902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1091/2185] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:17.949653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09765b93-a4f1-44f3-a986-0d6cdf167cc8 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:17.957405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce70ee-52cf-4b61-8713-6abc3d46307f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:17.958184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce70ee-52cf-4b61-8713-6abc3d46307f service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:17.958468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce70ee-52cf-4b61-8713-6abc3d46307f service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:17.958743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce70ee-52cf-4b61-8713-6abc3d46307f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:17.973509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce70ee-52cf-4b61-8713-6abc3d46307f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:17.978652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce70ee-52cf-4b61-8713-6abc3d46307f service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:18.017295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1095/2186] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:30:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 11 16:30:19.490513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5d473-84df-4996-bc28-f976fc7155b3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:19.490996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5d473-84df-4996-bc28-f976fc7155b3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:19.490996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5d473-84df-4996-bc28-f976fc7155b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:19.509747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-75e5d473-84df-4996-bc28-f976fc7155b3 None None] MFA Rules not processed for user `992a34be52b7491da0b68cbe17f25b39`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:19.510952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-75e5d473-84df-4996-bc28-f976fc7155b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:19.517976 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1092/2187] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:20.266089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ee2b-1049-4c62-8063-0ba0801c19cf None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:20.266605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ee2b-1049-4c62-8063-0ba0801c19cf None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:20.266703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ee2b-1049-4c62-8063-0ba0801c19cf None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:20.268136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1096/2188] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:30:20.316130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b863-2614-457e-a4a0-bb5ea0b0939a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:20.316664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b863-2614-457e-a4a0-bb5ea0b0939a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:20.317215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51b863-2614-457e-a4a0-bb5ea0b0939a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:20.361433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-fb51b863-2614-457e-a4a0-bb5ea0b0939a None None] MFA Rules not processed for user `08abc209f6cd4b62b34cad53c8ba4c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:20.364748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fb51b863-2614-457e-a4a0-bb5ea0b0939a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:20.395853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6c07-fb09-4273-9284-eb3942ea0758 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:20.395984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6c07-fb09-4273-9284-eb3942ea0758 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:20.396313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6c07-fb09-4273-9284-eb3942ea0758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:20.427379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-341a6c07-fb09-4273-9284-eb3942ea0758 None None] MFA Rules not processed for user `e7e5b04101c1403d9b3a311803ac798f`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:20.429321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-341a6c07-fb09-4273-9284-eb3942ea0758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:20.487296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1093/2189] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:20.489085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b863-2614-457e-a4a0-bb5ea0b0939a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:20.490618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fb51b863-2614-457e-a4a0-bb5ea0b0939a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:20.511479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51b863-2614-457e-a4a0-bb5ea0b0939a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:20.516819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:20.517894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:20.518386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:20.519391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] PATH_INFO: `/v3/users/7d3bafbdaf274f33aef3c7d066eda6c8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:20.521302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3bafbdaf274f33aef3c7d066eda6c8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:20.523500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:20.560583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1097/2190] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:20.561996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a6c07-fb09-4273-9284-eb3942ea0758 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:20.566337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-341a6c07-fb09-4273-9284-eb3942ea0758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:20.579446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:20.600847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a6c07-fb09-4273-9284-eb3942ea0758 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:20.610873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2c115-e26f-4bae-b161-ffbdfd4555d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:20.611554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2c115-e26f-4bae-b161-ffbdfd4555d1 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:20.611793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2c115-e26f-4bae-b161-ffbdfd4555d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:20.612049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2c115-e26f-4bae-b161-ffbdfd4555d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:20.614001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3bafbdaf274f33aef3c7d066eda6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:20.633561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2c115-e26f-4bae-b161-ffbdfd4555d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:20.635168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2c115-e26f-4bae-b161-ffbdfd4555d1 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:20.638512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3bafbdaf274f33aef3c7d066eda6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:20.641008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3bafbdaf274f33aef3c7d066eda6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:20.646672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1098/2191] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:20.681151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3bafbdaf274f33aef3c7d066eda6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:20.682547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-63421370-398d-423c-b99b-51d9e683e54c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3bafbdaf274f33aef3c7d066eda6c8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:20.718319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1094/2192] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:30:20 2022] DELETE /identity/v3/users/7d3bafbdaf274f33aef3c7d066eda6c8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:21.016494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2c115-e26f-4bae-b161-ffbdfd4555d1 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:21.036089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2c115-e26f-4bae-b161-ffbdfd4555d1 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:21.094109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:21.125509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:21.126148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.126427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.126999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] PATH_INFO: `/v3/projects/26ff1514f2c240f0808f8aad188ac97f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.128799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ff1514f2c240f0808f8aad188ac97f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:21.131163 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:21.180590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ff1514f2c240f0808f8aad188ac97f', 'request_id': 'req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:21.187359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ff1514f2c240f0808f8aad188ac97f', 'request_id': 'req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:21.194314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ff1514f2c240f0808f8aad188ac97f', 'request_id': 'req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:21.213443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] The token cache is being invalidate because project 26ff1514f2c240f0808f8aad188ac97f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:30:21.213740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:21.214396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a2b1eef9-111e-47a4-b82d-e2832f8ae8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:30:21.215885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1099/2193] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:30:21 2022] DELETE /identity/v3/projects/26ff1514f2c240f0808f8aad188ac97f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:21.222103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0dc3f-81b6-4117-a733-386f96b931cd None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.222427 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0dc3f-81b6-4117-a733-386f96b931cd None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.223482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0dc3f-81b6-4117-a733-386f96b931cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.249694 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0ab0dc3f-81b6-4117-a733-386f96b931cd None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:21.251430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0ab0dc3f-81b6-4117-a733-386f96b931cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:21.329869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1095/2194] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:30:21.338269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7d9fe-062a-4649-8641-060a863c5c1e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.338792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7d9fe-062a-4649-8641-060a863c5c1e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.338890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7d9fe-062a-4649-8641-060a863c5c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.358943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-85f7d9fe-062a-4649-8641-060a863c5c1e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:21.360280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-85f7d9fe-062a-4649-8641-060a863c5c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:21.368982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1100/2195] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:21.373362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0dc3f-81b6-4117-a733-386f96b931cd None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:21.384761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0dc3f-81b6-4117-a733-386f96b931cd None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:21.389806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:21.390420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.390702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.390975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.391621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:30:21.391923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:21.393564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:21.405900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1096/2196] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:30:21.410424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7d9fe-062a-4649-8641-060a863c5c1e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:21.422883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7d9fe-062a-4649-8641-060a863c5c1e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:21.430013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c203b-0232-4a31-888b-444752edafe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:21.430637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c203b-0232-4a31-888b-444752edafe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.430913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c203b-0232-4a31-888b-444752edafe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.431181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c203b-0232-4a31-888b-444752edafe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.432076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c203b-0232-4a31-888b-444752edafe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:21.434005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c203b-0232-4a31-888b-444752edafe7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:21.469091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1101/2197] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:30:21 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:30:21.472813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:21.492645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c36b8-45e9-46bc-a948-ea2a9c7ecc04 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:21.580695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c67a8-32db-4633-9acc-dcea32e0a190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:21.581989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c67a8-32db-4633-9acc-dcea32e0a190 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.582402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c67a8-32db-4633-9acc-dcea32e0a190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.582688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c67a8-32db-4633-9acc-dcea32e0a190 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.583343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c67a8-32db-4633-9acc-dcea32e0a190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:21.585386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c67a8-32db-4633-9acc-dcea32e0a190 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:21.625023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1097/2198] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:30:21 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:30:21.629854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c203b-0232-4a31-888b-444752edafe7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:21.641058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c203b-0232-4a31-888b-444752edafe7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:21.646498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea424b5-1dab-4452-955b-311c1f4fceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:21.647109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea424b5-1dab-4452-955b-311c1f4fceb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.647402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea424b5-1dab-4452-955b-311c1f4fceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.647662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea424b5-1dab-4452-955b-311c1f4fceb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.648342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea424b5-1dab-4452-955b-311c1f4fceb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:21.649933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea424b5-1dab-4452-955b-311c1f4fceb9 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:21.661981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1102/2199] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:30:21.666171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c67a8-32db-4633-9acc-dcea32e0a190 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:21.676910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c67a8-32db-4633-9acc-dcea32e0a190 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:21.682462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacd706-f397-4938-9531-72ec37bfa0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:21.683192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacd706-f397-4938-9531-72ec37bfa0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.683829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacd706-f397-4938-9531-72ec37bfa0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.683942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacd706-f397-4938-9531-72ec37bfa0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.684609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacd706-f397-4938-9531-72ec37bfa0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:21.686220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacd706-f397-4938-9531-72ec37bfa0e4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:21.702632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1098/2200] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:30:21.704211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea424b5-1dab-4452-955b-311c1f4fceb9 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:21.719308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea424b5-1dab-4452-955b-311c1f4fceb9 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:21.724132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:21.724790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.725066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.725338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] PATH_INFO: `/v3/projects/0550e07d9e474e5485aa01188de165b8/users/b7831d3847004751880cdda8d2e6ef27/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.745201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0550e07d9e474e5485aa01188de165b8, user_id=b7831d3847004751880cdda8d2e6ef27, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:21.749771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:21.767933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1103/2201] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:30:21 2022] PUT /identity/v3/projects/0550e07d9e474e5485aa01188de165b8/users/b7831d3847004751880cdda8d2e6ef27/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:30:21.779492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752574b-9773-4292-836e-942dd36c30df None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:21.779775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752574b-9773-4292-836e-942dd36c30df None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:21.780329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752574b-9773-4292-836e-942dd36c30df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:21.823636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:21.827692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c752574b-9773-4292-836e-942dd36c30df None None] MFA Rules not processed for user `b7831d3847004751880cdda8d2e6ef27`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:21.830759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c752574b-9773-4292-836e-942dd36c30df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:21.954886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1099/2202] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:21.966181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:21.967895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:22.027573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093689d4-fa8c-4cac-b6f5-aa56cbb9d28b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:22.031200 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:22.031809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:22.032089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:22.032396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:22.046620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:22.048067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:22.059046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1104/2203] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:30:23.428834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf59b6-05b4-4f5c-ae59-a03d40e47726 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:23.429374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf59b6-05b4-4f5c-ae59-a03d40e47726 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:23.429374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf59b6-05b4-4f5c-ae59-a03d40e47726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:23.585293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4ccf59b6-05b4-4f5c-ae59-a03d40e47726 None None] MFA Rules not processed for user `b7831d3847004751880cdda8d2e6ef27`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:23.586511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4ccf59b6-05b4-4f5c-ae59-a03d40e47726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:23.594124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1100/2204] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:32.054919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:33.579411 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=29 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:35.084634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:35.086689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:35.184884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74bc5-f7ab-49c9-bc4c-6358920d5030 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:35.185785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74bc5-f7ab-49c9-bc4c-6358920d5030 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:35.187137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74bc5-f7ab-49c9-bc4c-6358920d5030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:35.204318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c82c5-54f2-4a23-8013-afceb26dede1 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:35.208937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:35.210135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:35.210421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:35.210691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:35.212984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:35.217628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d6f74bc5-f7ab-49c9-bc4c-6358920d5030 None None] MFA Rules not processed for user `5d59680b3fde47bf81c7765ca4d8a757`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:35.219051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d6f74bc5-f7ab-49c9-bc4c-6358920d5030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:35.312191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:35.314087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:35.320313 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1101/2205] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:30:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:35.326004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1105/2206] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:30:38.932621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd4f7-9bcf-4b07-930d-42afb7a6691e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:38.933190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd4f7-9bcf-4b07-930d-42afb7a6691e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:38.933917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efd4f7-9bcf-4b07-930d-42afb7a6691e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:38.970249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-11efd4f7-9bcf-4b07-930d-42afb7a6691e None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:38.972388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-11efd4f7-9bcf-4b07-930d-42afb7a6691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:39.109652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1102/2207] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:30:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:39.127208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:39.131676 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:39.366700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087b7d7-8e19-487a-8cb8-697a3c937cc2 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:39.385193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4792-1b04-4c55-a648-fa8f68e24083 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:39.386095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4792-1b04-4c55-a648-fa8f68e24083 service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:39.386578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4792-1b04-4c55-a648-fa8f68e24083 service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:39.387050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4792-1b04-4c55-a648-fa8f68e24083 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:39.396878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea4792-1b04-4c55-a648-fa8f68e24083 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:39.398838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea4792-1b04-4c55-a648-fa8f68e24083 service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:39.408360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1106/2208] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:30:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:30:49.099862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=50 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:49.405044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=82 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:30:58.632235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911387a-889e-483e-89f7-097a0182e826 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:58.632759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911387a-889e-483e-89f7-097a0182e826 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:58.632759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911387a-889e-483e-89f7-097a0182e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:58.652999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6911387a-889e-483e-89f7-097a0182e826 None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:58.654214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6911387a-889e-483e-89f7-097a0182e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:58.661393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1103/2209] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:30:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:58.687046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4792-1b04-4c55-a648-fa8f68e24083 service swift] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:58.698679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4792-1b04-4c55-a648-fa8f68e24083 service swift] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:58.704227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd804385-2c77-4da9-a67e-34a0130dbf63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:58.704866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd804385-2c77-4da9-a67e-34a0130dbf63 service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:58.705681 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd804385-2c77-4da9-a67e-34a0130dbf63 service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:58.706035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd804385-2c77-4da9-a67e-34a0130dbf63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:58.714341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd804385-2c77-4da9-a67e-34a0130dbf63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:58.715926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd804385-2c77-4da9-a67e-34a0130dbf63 service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:58.724411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1107/2210] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:30:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:30:59.354937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb211b-ce41-4a3e-a0d1-bac3b7c57150 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:59.355222 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb211b-ce41-4a3e-a0d1-bac3b7c57150 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:59.355550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb211b-ce41-4a3e-a0d1-bac3b7c57150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:59.382649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b0eb211b-ce41-4a3e-a0d1-bac3b7c57150 None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:59.384283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b0eb211b-ce41-4a3e-a0d1-bac3b7c57150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:59.393495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1104/2211] 10.222.0.43 () {58 vars in 967 bytes} [Thu Aug 11 16:30:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:30:59.410838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd804385-2c77-4da9-a67e-34a0130dbf63 service swift] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:59.422984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd804385-2c77-4da9-a67e-34a0130dbf63 service swift] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:59.429755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bea0a1-8b11-4d84-bc10-4485e625d200 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:59.430365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea0a1-8b11-4d84-bc10-4485e625d200 service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:59.430648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea0a1-8b11-4d84-bc10-4485e625d200 service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:59.430918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bea0a1-8b11-4d84-bc10-4485e625d200 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:59.439744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bea0a1-8b11-4d84-bc10-4485e625d200 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:59.442865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bea0a1-8b11-4d84-bc10-4485e625d200 service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:59.462363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1108/2212] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:30:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 11 16:30:59.690950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068f18b-015b-44ab-a778-8ddc7de9b5d6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:59.691442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068f18b-015b-44ab-a778-8ddc7de9b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:59.691661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068f18b-015b-44ab-a778-8ddc7de9b5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:59.715929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e068f18b-015b-44ab-a778-8ddc7de9b5d6 None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:59.717525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e068f18b-015b-44ab-a778-8ddc7de9b5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:59.725885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1105/2213] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:30:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:30:59.740832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bea0a1-8b11-4d84-bc10-4485e625d200 service swift] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:59.751700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bea0a1-8b11-4d84-bc10-4485e625d200 service swift] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:59.757890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54cb15-dbf9-483d-81d0-e026f1b65c95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:59.758297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54cb15-dbf9-483d-81d0-e026f1b65c95 service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:59.758749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54cb15-dbf9-483d-81d0-e026f1b65c95 service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:59.759022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54cb15-dbf9-483d-81d0-e026f1b65c95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:59.768515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54cb15-dbf9-483d-81d0-e026f1b65c95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:59.770509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54cb15-dbf9-483d-81d0-e026f1b65c95 service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:30:59.780744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1109/2214] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:30:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:30:59.786540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951afb-af28-4678-9f71-d2ab2ad8dc5f None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:59.786816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951afb-af28-4678-9f71-d2ab2ad8dc5f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:59.787086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951afb-af28-4678-9f71-d2ab2ad8dc5f None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:59.788872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1106/2215] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:30:59.795067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522028b-5f13-4632-8208-75e6c2d42829 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:59.795359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522028b-5f13-4632-8208-75e6c2d42829 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:59.795633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522028b-5f13-4632-8208-75e6c2d42829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:59.816974 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e522028b-5f13-4632-8208-75e6c2d42829 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:30:59.818202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e522028b-5f13-4632-8208-75e6c2d42829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:59.901439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1110/2216] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:30:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:30:59.912818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951afb-af28-4678-9f71-d2ab2ad8dc5f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:30:59.914722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-87951afb-af28-4678-9f71-d2ab2ad8dc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:30:59.978575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951afb-af28-4678-9f71-d2ab2ad8dc5f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:30:59.982391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef62d7-5156-455a-823c-08f6cd29ac4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7a2237c18934cb2aa3699a5776874ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:30:59.983003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef62d7-5156-455a-823c-08f6cd29ac4f service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:30:59.983277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef62d7-5156-455a-823c-08f6cd29ac4f service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:30:59.983662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef62d7-5156-455a-823c-08f6cd29ac4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:30:59.994967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef62d7-5156-455a-823c-08f6cd29ac4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:30:59.996442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef62d7-5156-455a-823c-08f6cd29ac4f service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:00.007966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1107/2217] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:30:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:31:00.085610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8968c6-fa54-4f59-9264-de999aec5d06 None None] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:00.085898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8968c6-fa54-4f59-9264-de999aec5d06 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:00.086226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8968c6-fa54-4f59-9264-de999aec5d06 None None] PATH_INFO: `/` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:00.087638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1111/2218] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:31:00.092378 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d4d7-af0a-4dfa-bf49-9ea3e8fe3c0e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:00.092664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d4d7-af0a-4dfa-bf49-9ea3e8fe3c0e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:00.092938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25d4d7-af0a-4dfa-bf49-9ea3e8fe3c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:00.112724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9e25d4d7-af0a-4dfa-bf49-9ea3e8fe3c0e None None] MFA Rules not processed for user `123d33bb6caa464aba4f9be6b0fb7656`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:31:00.113973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9e25d4d7-af0a-4dfa-bf49-9ea3e8fe3c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:00.120546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1108/2219] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:31:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:31:00.133500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8968c6-fa54-4f59-9264-de999aec5d06 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:00.134801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1b8968c6-fa54-4f59-9264-de999aec5d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:00.148503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8968c6-fa54-4f59-9264-de999aec5d06 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:00.152165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2e1f8-94f4-4831-a6e5-849d1570381e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:00.152741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2e1f8-94f4-4831-a6e5-849d1570381e service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:00.153014 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2e1f8-94f4-4831-a6e5-849d1570381e service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:00.153279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2e1f8-94f4-4831-a6e5-849d1570381e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:00.161218 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2e1f8-94f4-4831-a6e5-849d1570381e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:00.162637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2e1f8-94f4-4831-a6e5-849d1570381e service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:00.163925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55e76c-96cf-4e4f-94fd-2cd4730b95dc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:00.164359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55e76c-96cf-4e4f-94fd-2cd4730b95dc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:00.164634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55e76c-96cf-4e4f-94fd-2cd4730b95dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:00.170840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1112/2220] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:31:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:31:00.184785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8a55e76c-96cf-4e4f-94fd-2cd4730b95dc None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:31:00.185978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8a55e76c-96cf-4e4f-94fd-2cd4730b95dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:00.194366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1109/2221] 10.222.0.43 () {58 vars in 967 bytes} [Thu Aug 11 16:31:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:31:00.207011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2e1f8-94f4-4831-a6e5-849d1570381e service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:00.218972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2e1f8-94f4-4831-a6e5-849d1570381e service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:00.224365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e47c0-04be-456c-b61a-aa762b1762e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:00.224958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e47c0-04be-456c-b61a-aa762b1762e7 service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:00.225232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e47c0-04be-456c-b61a-aa762b1762e7 service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:00.225498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e47c0-04be-456c-b61a-aa762b1762e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:00.234061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e47c0-04be-456c-b61a-aa762b1762e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:00.235611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e47c0-04be-456c-b61a-aa762b1762e7 service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:00.244604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1113/2222] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:31:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:31:00.262170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a4585-bcaa-423d-b5a4-31ad022ee654 None None] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:00.262690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a4585-bcaa-423d-b5a4-31ad022ee654 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:00.263143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a4585-bcaa-423d-b5a4-31ad022ee654 None None] PATH_INFO: `/` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:00.264582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1110/2223] 10.222.0.43 () {54 vars in 921 bytes} [Thu Aug 11 16:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 11 16:31:00.270673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875bd2f-d826-4dcc-b61d-13c9718fa530 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:00.270962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875bd2f-d826-4dcc-b61d-13c9718fa530 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:00.271243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875bd2f-d826-4dcc-b61d-13c9718fa530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:00.295093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-0875bd2f-d826-4dcc-b61d-13c9718fa530 None None] MFA Rules not processed for user `2e3819e78802469c83c09a1cdd865f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:31:00.296391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0875bd2f-d826-4dcc-b61d-13c9718fa530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:00.309153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1114/2224] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Aug 11 16:31:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:31:00.319980 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a4585-bcaa-423d-b5a4-31ad022ee654 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:00.331934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a4585-bcaa-423d-b5a4-31ad022ee654 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:00.335655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7922b-b4c6-46c9-a80b-214a66f51bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:00.336299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7922b-b4c6-46c9-a80b-214a66f51bec service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:00.336577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7922b-b4c6-46c9-a80b-214a66f51bec service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:00.336844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7922b-b4c6-46c9-a80b-214a66f51bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:00.349274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7922b-b4c6-46c9-a80b-214a66f51bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:00.350812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7922b-b4c6-46c9-a80b-214a66f51bec service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:00.362476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1111/2225] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:31:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:31:02.370911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875bd2f-d826-4dcc-b61d-13c9718fa530 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:02.442489 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875bd2f-d826-4dcc-b61d-13c9718fa530 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:02.447452 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:02.448088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:02.448400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:02.448674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] PATH_INFO: `/v3/users/b7831d3847004751880cdda8d2e6ef27` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:02.449825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7831d3847004751880cdda8d2e6ef27)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:02.451739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:02.475814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:31:02.505215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7831d3847004751880cdda8d2e6ef27'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:02.519029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7831d3847004751880cdda8d2e6ef27'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:02.519869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7831d3847004751880cdda8d2e6ef27'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:02.527687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7831d3847004751880cdda8d2e6ef27'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:02.536788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7831d3847004751880cdda8d2e6ef27'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:02.543445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1115/2226] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:31:02 2022] DELETE /identity/v3/users/b7831d3847004751880cdda8d2e6ef27 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:31:02.548347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707dd4f-fd66-4b8a-b042-a7077748d004 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:02.548881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707dd4f-fd66-4b8a-b042-a7077748d004 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:02.549088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707dd4f-fd66-4b8a-b042-a7077748d004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:02.569122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f707dd4f-fd66-4b8a-b042-a7077748d004 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:31:02.570359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f707dd4f-fd66-4b8a-b042-a7077748d004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:02.647901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1112/2227] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:31:02.660151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:02.661868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:02.725491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06536b-0b5f-43c6-aa73-a03bfa77a859 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:02.731488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213c842-4754-4e18-89c1-bd4020aecb02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:02.732051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213c842-4754-4e18-89c1-bd4020aecb02 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:02.732451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213c842-4754-4e18-89c1-bd4020aecb02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:02.732881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213c842-4754-4e18-89c1-bd4020aecb02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:02.746212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213c842-4754-4e18-89c1-bd4020aecb02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:02.747732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213c842-4754-4e18-89c1-bd4020aecb02 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:02.762724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1116/2228] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:31:02.993751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707dd4f-fd66-4b8a-b042-a7077748d004 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:02.995774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f707dd4f-fd66-4b8a-b042-a7077748d004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:03.010955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707dd4f-fd66-4b8a-b042-a7077748d004 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:03.018708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:03.019382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:03.019644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:03.019916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] PATH_INFO: `/v3/projects/0550e07d9e474e5485aa01188de165b8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:03.021664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0550e07d9e474e5485aa01188de165b8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:03.023004 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:03.085585 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0550e07d9e474e5485aa01188de165b8', 'request_id': 'req-8dd76c19-ec6a-4395-802c-af9699a61184'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:03.088349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:31:03.094126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0550e07d9e474e5485aa01188de165b8', 'request_id': 'req-8dd76c19-ec6a-4395-802c-af9699a61184'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:03.101916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0550e07d9e474e5485aa01188de165b8', 'request_id': 'req-8dd76c19-ec6a-4395-802c-af9699a61184'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:03.112683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] The token cache is being invalidate because project 0550e07d9e474e5485aa01188de165b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:31:03.112909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:03.113576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:31:03.115029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1113/2229] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:31:02 2022] DELETE /identity/v3/projects/0550e07d9e474e5485aa01188de165b8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:31:03.285735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad89b3-005a-4be2-89a6-27631018d062 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:03.286119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad89b3-005a-4be2-89a6-27631018d062 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:03.286464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad89b3-005a-4be2-89a6-27631018d062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:03.308270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-daad89b3-005a-4be2-89a6-27631018d062 None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:31:03.309513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-daad89b3-005a-4be2-89a6-27631018d062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:03.368991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1117/2230] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 16:31:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:31:03.388142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:03.389964 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:03.462402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd76c19-ec6a-4395-802c-af9699a61184 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:03.467405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e24c8-ee9d-4aeb-955f-79aaa02d558d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:03.468024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e24c8-ee9d-4aeb-955f-79aaa02d558d service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:03.468324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e24c8-ee9d-4aeb-955f-79aaa02d558d service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:03.468593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e24c8-ee9d-4aeb-955f-79aaa02d558d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:03.477476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e24c8-ee9d-4aeb-955f-79aaa02d558d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:03.479120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e24c8-ee9d-4aeb-955f-79aaa02d558d service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:03.488482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1114/2231] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:31:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:31:13.361067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:DELETE=7,keystone:SELECT=71,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:31:13.484117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=3,keystone:SELECT=38 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:31:22.354675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad89b3-005a-4be2-89a6-27631018d062 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:22.356070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-daad89b3-005a-4be2-89a6-27631018d062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:22.428045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad89b3-005a-4be2-89a6-27631018d062 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:22.434010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:22.434615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:22.435506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:22.435787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:22.437593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:22.450313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:22.451911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:22.463098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1118/2232] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:31:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:31:32.457651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=32 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:31:32.747745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11a3c7-cd69-4a6f-9990-7107e60f1aaf None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:32.748280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11a3c7-cd69-4a6f-9990-7107e60f1aaf None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:32.748380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11a3c7-cd69-4a6f-9990-7107e60f1aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:32.767940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cd11a3c7-cd69-4a6f-9990-7107e60f1aaf None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:31:32.769233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cd11a3c7-cd69-4a6f-9990-7107e60f1aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:32.777009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1115/2233] 10.222.0.43 () {58 vars in 967 bytes} [Thu Aug 11 16:31:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:31:32.788985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:31:32.799922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676093d-1b45-4813-bc66-c22ada21759d service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:31:32.804906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5443980-24ba-4978-b718-de19af97b25b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:31:32.805496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5443980-24ba-4978-b718-de19af97b25b service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:32.805820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5443980-24ba-4978-b718-de19af97b25b service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:32.806090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5443980-24ba-4978-b718-de19af97b25b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:32.815434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5443980-24ba-4978-b718-de19af97b25b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:31:32.817208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5443980-24ba-4978-b718-de19af97b25b service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:31:32.825989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1119/2234] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 16:31:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 16:31:42.763390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:31:42.822778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=6 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:31:49.010045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99eaa0-bf3d-4d51-8993-d7b4b5b9a7b4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:31:49.010599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99eaa0-bf3d-4d51-8993-d7b4b5b9a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:31:49.010599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99eaa0-bf3d-4d51-8993-d7b4b5b9a7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:31:49.029464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2a99eaa0-bf3d-4d51-8993-d7b4b5b9a7b4 None None] MFA Rules not processed for user `992a34be52b7491da0b68cbe17f25b39`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:31:49.030657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2a99eaa0-bf3d-4d51-8993-d7b4b5b9a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:31:49.089089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1116/2235] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:31:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:31:59.082167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=25 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:32:19.381965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659ee3-d0a8-473e-913f-e4cf2a552ced None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:32:19.382493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659ee3-d0a8-473e-913f-e4cf2a552ced None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:32:19.382493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af659ee3-d0a8-473e-913f-e4cf2a552ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:32:19.403158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-af659ee3-d0a8-473e-913f-e4cf2a552ced None None] MFA Rules not processed for user `e7e5b04101c1403d9b3a311803ac798f`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:32:19.404384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-af659ee3-d0a8-473e-913f-e4cf2a552ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:32:19.457034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1120/2236] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:32:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:32:29.450595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=25 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:32:55.527160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99eaa0-bf3d-4d51-8993-d7b4b5b9a7b4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:32:55.528719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2a99eaa0-bf3d-4d51-8993-d7b4b5b9a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:32:55.594190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99eaa0-bf3d-4d51-8993-d7b4b5b9a7b4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:32:55.597942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:32:55.598657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:32:55.598920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:32:55.599185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:32:55.600898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:32:55.669756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:32:55.671152 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:32:55.681642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1117/2237] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:32:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:32:56.788144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af659ee3-d0a8-473e-913f-e4cf2a552ced None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:32:56.789644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-af659ee3-d0a8-473e-913f-e4cf2a552ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:32:56.801993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af659ee3-d0a8-473e-913f-e4cf2a552ced None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:32:56.806214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:32:56.806816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:32:56.807095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:32:56.807365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:32:56.809348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:32:56.823202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:32:56.824667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:32:56.835318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1121/2238] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:32:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:04.000595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:04.002010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:04.017154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d743cdf-6f03-44ad-8e2f-3afd711a8bcc service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:04.024200 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:04.024819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:04.025213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:04.025489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:04.028525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:04.095990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:04.097581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:04.106975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1118/2239] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:06.830629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=6 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:33:14.102240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=82 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:33:15.628577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:15.628577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:15.629126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:15.629126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:15.639127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8cfa4-3466-470c-9aa6-4ad2cf371cea service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:15.645437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:15.646069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:15.646350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:15.646626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:15.648491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:15.686549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af22092-9ba9-46ec-9f99-0b1ef87c8462 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:15.690474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:15.691090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:15.691370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:15.691644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:15.693432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:15.735468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:15.737303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:15.750000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1119/2240] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:33:15.761452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:15.765478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:15.772036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:15.776260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:15.784964 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99116f69-abd9-40b5-8bdf-2da271719406 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:15.790001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92440b85-c159-4cf3-bcc9-b933d4d3fd61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:15.790880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440b85-c159-4cf3-bcc9-b933d4d3fd61 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:15.791369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440b85-c159-4cf3-bcc9-b933d4d3fd61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:15.792428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92440b85-c159-4cf3-bcc9-b933d4d3fd61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:15.794481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-92440b85-c159-4cf3-bcc9-b933d4d3fd61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:15.803553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1122/2241] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:15.870774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92440b85-c159-4cf3-bcc9-b933d4d3fd61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:15.872251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92440b85-c159-4cf3-bcc9-b933d4d3fd61 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:15.881825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1120/2242] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:19.443539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:19.457703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46a926-739b-428a-8e93-69c2631504e4 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:19.463464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:19.465039 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:19.465380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:19.466011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:19.468524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:19.484793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:19.486466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:19.497549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1123/2243] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:21.747945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8c309-3583-4656-a3b7-a018ace05a99 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:21.748531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8c309-3583-4656-a3b7-a018ace05a99 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:21.748531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8c309-3583-4656-a3b7-a018ace05a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:21.769527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-22d8c309-3583-4656-a3b7-a018ace05a99 None None] MFA Rules not processed for user `3a5ab962b58f473f874bd94bfe2db6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:33:21.770826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-22d8c309-3583-4656-a3b7-a018ace05a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:21.778590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1121/2244] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:33:22.089938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:22.099221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37945ad-8d49-4a8e-a921-4f768d6d39ad service nova] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:22.103223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:22.104675 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:22.105131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:22.105562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:22.107632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:22.121463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:22.121463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:22.129380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1124/2245] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:22.435959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8c309-3583-4656-a3b7-a018ace05a99 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:22.437402 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-22d8c309-3583-4656-a3b7-a018ace05a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:22.452784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8c309-3583-4656-a3b7-a018ace05a99 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:22.457932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd054483-1b79-485e-bd66-7d9a90886932 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:22.458539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054483-1b79-485e-bd66-7d9a90886932 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:22.458812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054483-1b79-485e-bd66-7d9a90886932 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:22.459078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd054483-1b79-485e-bd66-7d9a90886932 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:22.460782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dd054483-1b79-485e-bd66-7d9a90886932 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:22.473371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd054483-1b79-485e-bd66-7d9a90886932 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:22.475015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd054483-1b79-485e-bd66-7d9a90886932 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:22.485532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1122/2246] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:26.424259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:26.435090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5cfc-5fd4-4ba1-bf10-7f84d4d0918d service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:26.438870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:26.439469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:26.440069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:26.440231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:26.442259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:26.455709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:26.457186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:26.466577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1125/2247] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:32.481348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=65 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:33:32.575441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0ac37-9dc5-470a-9ddc-0563a581c3d3 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:32.575722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0ac37-9dc5-470a-9ddc-0563a581c3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:32.575996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0ac37-9dc5-470a-9ddc-0563a581c3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:32.595039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-51b0ac37-9dc5-470a-9ddc-0563a581c3d3 None None] MFA Rules not processed for user `5d59680b3fde47bf81c7765ca4d8a757`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:33:32.596241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-51b0ac37-9dc5-470a-9ddc-0563a581c3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:32.603021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1123/2248] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:33:35.317304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:33:35.330075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533b355-f421-46c2-a93b-5650070b29e6 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:33:35.335573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:33:35.336193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:33:35.336474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:33:35.336744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:33:35.338562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:33:35.350966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:33:35.352563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:33:35.363791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1126/2249] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:33:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:33:42.590237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:33:45.359542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=72 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:35:15.962575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0ac37-9dc5-470a-9ddc-0563a581c3d3 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:35:15.974843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0ac37-9dc5-470a-9ddc-0563a581c3d3 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:35:15.980168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:35:15.980765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:35:15.981042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:35:15.981312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:35:15.983162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:35:16.046392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:35:16.047988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:35:16.057862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1124/2250] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:35:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:35:18.780736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:35:18.790648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff535-40d4-4ff3-9610-bfa39d8d935f service nova] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:35:18.794244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a264d03-867b-4492-bddf-694afd2d7248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:35:18.794843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a264d03-867b-4492-bddf-694afd2d7248 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:35:18.795788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a264d03-867b-4492-bddf-694afd2d7248 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:35:18.796061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a264d03-867b-4492-bddf-694afd2d7248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:35:18.797979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7a264d03-867b-4492-bddf-694afd2d7248 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:35:18.809254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a264d03-867b-4492-bddf-694afd2d7248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:35:18.811117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a264d03-867b-4492-bddf-694afd2d7248 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:35:18.821982 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1127/2251] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:35:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:35:21.337380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:35:21.354001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d0508-a810-4379-91e4-6c80ef5adcfd service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:35:21.360973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:35:21.361862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:35:21.362344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:35:21.362804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:35:21.364797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:35:21.379405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:35:21.381942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:35:21.394287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1125/2252] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:35:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:35:26.274799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440af25-88a7-4a1d-8159-861ba7cc5c3f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:35:26.275284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440af25-88a7-4a1d-8159-861ba7cc5c3f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:35:26.275448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440af25-88a7-4a1d-8159-861ba7cc5c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:35:26.296451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-a440af25-88a7-4a1d-8159-861ba7cc5c3f None None] MFA Rules not processed for user `e7e5b04101c1403d9b3a311803ac798f`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:35:26.297827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a440af25-88a7-4a1d-8159-861ba7cc5c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:35:26.305197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1128/2253] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:35:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:35:31.388877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=34 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:35:34.630410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:35:34.643599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1fdcc1-83e4-4282-9f2f-66ace107040d service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:35:34.649168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:35:34.649776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:35:34.650053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:35:34.650339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:35:34.652075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:35:34.664499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:35:34.666060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:35:34.676944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1126/2254] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:35:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:35:36.290709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=9 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:35:44.672470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:37:12.834808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440af25-88a7-4a1d-8159-861ba7cc5c3f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:37:12.894895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440af25-88a7-4a1d-8159-861ba7cc5c3f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:37:12.900021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a62cec-621e-465e-9a1b-9cf5210b6a62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:37:12.900652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a62cec-621e-465e-9a1b-9cf5210b6a62 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:37:12.900933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a62cec-621e-465e-9a1b-9cf5210b6a62 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:37:12.901205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a62cec-621e-465e-9a1b-9cf5210b6a62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:37:12.910248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a62cec-621e-465e-9a1b-9cf5210b6a62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:37:12.911826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a62cec-621e-465e-9a1b-9cf5210b6a62 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:37:12.921497 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1129/2255] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:37:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:37:22.917421 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=27 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:37:57.584748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:37:57.621092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aaae2e-d3a4-44e0-9a37-8b784d0db431 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:37:57.624594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bc5b0-1ceb-486c-8829-509ce16b790a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:37:57.625179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bc5b0-1ceb-486c-8829-509ce16b790a service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:37:57.625449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bc5b0-1ceb-486c-8829-509ce16b790a service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:37:57.625713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bc5b0-1ceb-486c-8829-509ce16b790a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:37:57.642046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bc5b0-1ceb-486c-8829-509ce16b790a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:37:57.643461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bc5b0-1ceb-486c-8829-509ce16b790a service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:37:57.663867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1127/2256] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:37:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:38:07.661352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=25 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:38:16.434711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a62cec-621e-465e-9a1b-9cf5210b6a62 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:38:16.447303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a62cec-621e-465e-9a1b-9cf5210b6a62 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:38:16.452787 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff5a05-eede-49e4-ba04-6a5407320e1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:38:16.453375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff5a05-eede-49e4-ba04-6a5407320e1f service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:38:16.453648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff5a05-eede-49e4-ba04-6a5407320e1f service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:38:16.453918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff5a05-eede-49e4-ba04-6a5407320e1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:38:16.461846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff5a05-eede-49e4-ba04-6a5407320e1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:38:16.463397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff5a05-eede-49e4-ba04-6a5407320e1f service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:38:16.490418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1130/2257] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:38:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:38:17.092817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bc5b0-1ceb-486c-8829-509ce16b790a service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:38:17.104325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bc5b0-1ceb-486c-8829-509ce16b790a service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:38:17.109766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b30d83-ca72-40d5-8fe9-15e8cd0fa286 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:38:17.110376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b30d83-ca72-40d5-8fe9-15e8cd0fa286 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:38:17.110649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b30d83-ca72-40d5-8fe9-15e8cd0fa286 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:38:17.110932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b30d83-ca72-40d5-8fe9-15e8cd0fa286 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:38:17.119859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b30d83-ca72-40d5-8fe9-15e8cd0fa286 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:38:17.121478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b30d83-ca72-40d5-8fe9-15e8cd0fa286 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:38:17.153779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1128/2258] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:38:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:38:26.487222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=13 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:38:27.150703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=13 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:39:17.068691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff5a05-eede-49e4-ba04-6a5407320e1f service nova] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:39:17.090318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff5a05-eede-49e4-ba04-6a5407320e1f service nova] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:39:17.093954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eddfc7-352d-4e8d-89a4-76fbb24a9aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:39:17.094548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eddfc7-352d-4e8d-89a4-76fbb24a9aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:39:17.094821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eddfc7-352d-4e8d-89a4-76fbb24a9aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:39:17.095088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eddfc7-352d-4e8d-89a4-76fbb24a9aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:39:17.106272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eddfc7-352d-4e8d-89a4-76fbb24a9aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:39:17.107703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eddfc7-352d-4e8d-89a4-76fbb24a9aa6 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:39:17.119528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1131/2259] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:39:17.125262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b30d83-ca72-40d5-8fe9-15e8cd0fa286 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:39:17.135967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b30d83-ca72-40d5-8fe9-15e8cd0fa286 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:39:17.139457 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f8b6-d1cf-4dbb-aae2-1037f70d971d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:39:17.140045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f8b6-d1cf-4dbb-aae2-1037f70d971d service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:39:17.140362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f8b6-d1cf-4dbb-aae2-1037f70d971d service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:39:17.140608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03f8b6-d1cf-4dbb-aae2-1037f70d971d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:39:17.148203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03f8b6-d1cf-4dbb-aae2-1037f70d971d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:39:17.149603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03f8b6-d1cf-4dbb-aae2-1037f70d971d service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:39:17.163092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1129/2260] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:39:17.167158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eddfc7-352d-4e8d-89a4-76fbb24a9aa6 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:39:17.178299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eddfc7-352d-4e8d-89a4-76fbb24a9aa6 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:39:17.188417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec138-d8b8-4165-a225-2572010e01c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:39:17.189115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec138-d8b8-4165-a225-2572010e01c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:39:17.189413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec138-d8b8-4165-a225-2572010e01c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:39:17.189669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec138-d8b8-4165-a225-2572010e01c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:39:17.202122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec138-d8b8-4165-a225-2572010e01c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:39:17.204079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec138-d8b8-4165-a225-2572010e01c1 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:39:17.240032 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1132/2261] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:39:17.285520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f8b6-d1cf-4dbb-aae2-1037f70d971d service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:39:17.296946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03f8b6-d1cf-4dbb-aae2-1037f70d971d service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:39:17.300648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541312c1-d4c9-4b08-b82b-ae1554e5dd26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:39:17.301318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541312c1-d4c9-4b08-b82b-ae1554e5dd26 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:39:17.301693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541312c1-d4c9-4b08-b82b-ae1554e5dd26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:39:17.302024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541312c1-d4c9-4b08-b82b-ae1554e5dd26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:39:17.323769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541312c1-d4c9-4b08-b82b-ae1554e5dd26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:39:17.325278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541312c1-d4c9-4b08-b82b-ae1554e5dd26 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:39:17.358826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1130/2262] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:39:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:39:27.236160 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=23 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:39:27.354620 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=19 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:41:17.780479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541312c1-d4c9-4b08-b82b-ae1554e5dd26 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:41:17.781029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec138-d8b8-4165-a225-2572010e01c1 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:41:17.789968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec138-d8b8-4165-a225-2572010e01c1 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:41:17.792664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541312c1-d4c9-4b08-b82b-ae1554e5dd26 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:41:17.794343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:41:17.795005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:41:17.795482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:41:17.795758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:41:17.799385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:41:17.800454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:41:17.800737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:41:17.801012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:41:17.805383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:41:17.807734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:41:17.810302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:41:17.811963 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:41:17.838198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1133/2263] 10.222.0.43 () {60 vars in 1396 bytes} [Thu Aug 11 16:41:17 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:41:17.842248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1131/2264] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:41:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:41:27.834624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=13 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:41:27.839429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=13 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:42:17.825730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:42:17.827323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:42:17.921524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1065340-06cb-4ad2-9af6-1425fb88ecc8 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:42:17.926829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:42:17.927429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:42:17.927709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:42:17.927980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:42:17.929837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:42:17.941556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:42:17.943152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:42:17.952665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1134/2265] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:42:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:42:27.948634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=41 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:18.146904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:18.148536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:18.221519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9942819-7e29-4373-8b49-8d4baefd0d32 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:18.225170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:18.225773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:18.226054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:18.226322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:18.228191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:18.312593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:18.314305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:18.324925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1132/2266] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:43:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:43:18.378186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:18.387374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62876b8-ee6c-4116-ac6c-0717f7cbd5e0 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:18.391045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:18.391906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:18.391906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:18.391906 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:18.394270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:18.464264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:18.465680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:18.475046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1135/2267] 10.222.0.43 () {60 vars in 1396 bytes} [Thu Aug 11 16:43:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:43:18.607789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:18.617561 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa591e-d142-4bc2-9bec-2493fc7ed8a2 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:18.621237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:18.621850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:18.622112 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:18.622477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:18.624333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:18.699074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:18.700527 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:18.709348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1133/2268] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:43:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:43:20.484753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:20.487566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:20.526105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:20.530083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:20.546966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:20.594405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a50cb-1d1f-4f14-8c1a-e9a8f0419cdb service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:20.598209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:20.598937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:20.599522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:20.599799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:20.601703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:20.611210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d4cb14-d541-4e29-9a40-cc2fa9824fed service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:20.616475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:20.617376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:20.617831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:20.618245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:20.620554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:20.699686 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:20.701297 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:20.708933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:20.710601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:20.719446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:20.727359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1136/2269] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:43:20.731328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1134/2270] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:43:25.971511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:25.973214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:25.991747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbb09c-b129-45c5-8612-11b15bdd412b service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.003343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.004311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.004944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.006542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.011850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.031093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.034258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:26.052275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1137/2271] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:43:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:43:26.088458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:26.090614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.104677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a599926-72cd-44f7-831a-d147973303bd service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.111315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.112340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.112829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.113962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] PATH_INFO: `/v3/users/e7e5b04101c1403d9b3a311803ac798f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.139308 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e5b04101c1403d9b3a311803ac798f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.142386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:26.203109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e5b04101c1403d9b3a311803ac798f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.228063 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e5b04101c1403d9b3a311803ac798f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.230630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e5b04101c1403d9b3a311803ac798f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.249929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e5b04101c1403d9b3a311803ac798f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.251309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e5b04101c1403d9b3a311803ac798f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.270446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:26.279670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1135/2272] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:43:26 2022] DELETE /identity/v3/users/e7e5b04101c1403d9b3a311803ac798f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:43:26.340369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:26.353113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ecfe4-6e31-46ab-bd92-f21f5cfab499 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.411927 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:26.413989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.440635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.441265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.442087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.442370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.444902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.458601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.460416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:26.472217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1138/2273] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:43:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:43:26.491665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535ae17-29f7-4827-9a96-f5d2c80319be admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.496923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.497796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.497934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.498145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] PATH_INFO: `/v3/users/3a5ab962b58f473f874bd94bfe2db6b3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.507435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:26.511628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.520801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a5ab962b58f473f874bd94bfe2db6b3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.523115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:26.526453 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758b16-ea03-4b3b-91d5-78a7be627da2 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.531083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.531708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.532045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.532703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] PATH_INFO: `/v3/projects/91ed4e8fe7284bd5863ba22aa552e60b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.534051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ed4e8fe7284bd5863ba22aa552e60b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.535956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:26.574207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a5ab962b58f473f874bd94bfe2db6b3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.582531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a5ab962b58f473f874bd94bfe2db6b3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.583196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a5ab962b58f473f874bd94bfe2db6b3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.587572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ed4e8fe7284bd5863ba22aa552e60b', 'request_id': 'req-57df2fa1-0426-435c-a226-daf87735bae4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.593856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ed4e8fe7284bd5863ba22aa552e60b', 'request_id': 'req-57df2fa1-0426-435c-a226-daf87735bae4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.595529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a5ab962b58f473f874bd94bfe2db6b3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.596303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-60405f03-de0b-436d-9ef9-e4c5470c5b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a5ab962b58f473f874bd94bfe2db6b3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.599447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ed4e8fe7284bd5863ba22aa552e60b', 'request_id': 'req-57df2fa1-0426-435c-a226-daf87735bae4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.608953 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1136/2274] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:43:26 2022] DELETE /identity/v3/users/3a5ab962b58f473f874bd94bfe2db6b3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:43:26.613064 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] The token cache is being invalidate because project 91ed4e8fe7284bd5863ba22aa552e60b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:43:26.613225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.613890 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-57df2fa1-0426-435c-a226-daf87735bae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.615544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1139/2275] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:43:26 2022] DELETE /identity/v3/projects/91ed4e8fe7284bd5863ba22aa552e60b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:43:26.621328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67431e41-7466-4ad7-9b1b-87a1ba8f7b53 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.621619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67431e41-7466-4ad7-9b1b-87a1ba8f7b53 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.621897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67431e41-7466-4ad7-9b1b-87a1ba8f7b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.687458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-67431e41-7466-4ad7-9b1b-87a1ba8f7b53 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:43:26.689591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-67431e41-7466-4ad7-9b1b-87a1ba8f7b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.722758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:26.761310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1137/2276] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:43:26.763609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561323af-b3f9-46cf-a762-48aa6ad25a8f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.763960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561323af-b3f9-46cf-a762-48aa6ad25a8f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.764334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561323af-b3f9-46cf-a762-48aa6ad25a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.785183 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-561323af-b3f9-46cf-a762-48aa6ad25a8f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:43:26.788381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-561323af-b3f9-46cf-a762-48aa6ad25a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.795149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1140/2277] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:43:26.800264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67431e41-7466-4ad7-9b1b-87a1ba8f7b53 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:26.812309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561323af-b3f9-46cf-a762-48aa6ad25a8f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:26.814942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-561323af-b3f9-46cf-a762-48aa6ad25a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.816010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67431e41-7466-4ad7-9b1b-87a1ba8f7b53 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.820771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.821975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.821975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.822160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.822880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:43:26.823168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.825513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:26.831181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561323af-b3f9-46cf-a762-48aa6ad25a8f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.836945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.837844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.838281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.841727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] PATH_INFO: `/v3/projects/6d15983e6d8e4ad6887105f88f187044` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.842297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1138/2278] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:43:26.842659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d15983e6d8e4ad6887105f88f187044)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.842842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:26.842956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:26.854453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7d84d-0e35-476c-9f38-4a91d1b17190 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.859168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6b7c-834e-4fe4-850d-f8368da9d6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.859766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6b7c-834e-4fe4-850d-f8368da9d6ab admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.860217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6b7c-834e-4fe4-850d-f8368da9d6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.860499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d6b7c-834e-4fe4-850d-f8368da9d6ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.861120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d6b7c-834e-4fe4-850d-f8368da9d6ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.863385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d6b7c-834e-4fe4-850d-f8368da9d6ab admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:26.892794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:26.894307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d15983e6d8e4ad6887105f88f187044', 'request_id': 'req-b221584b-ff61-4685-9690-79a199e476db'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.901466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d15983e6d8e4ad6887105f88f187044', 'request_id': 'req-b221584b-ff61-4685-9690-79a199e476db'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.902844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1139/2279] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:43:26 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:43:26.907421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6b7c-834e-4fe4-850d-f8368da9d6ab admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:26.909691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d15983e6d8e4ad6887105f88f187044', 'request_id': 'req-b221584b-ff61-4685-9690-79a199e476db'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.925006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] The token cache is being invalidate because project 6d15983e6d8e4ad6887105f88f187044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:43:26.925301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.925953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b221584b-ff61-4685-9690-79a199e476db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:43:26.927461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1141/2280] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:43:26 2022] DELETE /identity/v3/projects/6d15983e6d8e4ad6887105f88f187044 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:43:26.934699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230fbb-03e0-4f25-8621-fb2a2f18da10 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.934983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230fbb-03e0-4f25-8621-fb2a2f18da10 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.935255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5230fbb-03e0-4f25-8621-fb2a2f18da10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.958071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e5230fbb-03e0-4f25-8621-fb2a2f18da10 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:43:26.959766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e5230fbb-03e0-4f25-8621-fb2a2f18da10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:26.982522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d6b7c-834e-4fe4-850d-f8368da9d6ab admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:26.992612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:26.993318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:26.993667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:26.994033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:26.994754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:26.996870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.020329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1140/2281] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:43:26 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:43:27.029536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1142/2282] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:43:27.030019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.030019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:27.030893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8a60-45c5-4b4e-a73c-dd79fb892f5c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.031179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8a60-45c5-4b4e-a73c-dd79fb892f5c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.031455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8a60-45c5-4b4e-a73c-dd79fb892f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.038337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006beec-3994-4f39-b3c5-d2497bfd4020 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.047072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6bbb7-cb43-40be-88a7-9f1083bf4416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.048476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6bbb7-cb43-40be-88a7-9f1083bf4416 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.048758 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6bbb7-cb43-40be-88a7-9f1083bf4416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.049083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6bbb7-cb43-40be-88a7-9f1083bf4416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.049739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6bbb7-cb43-40be-88a7-9f1083bf4416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.054440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6bbb7-cb43-40be-88a7-9f1083bf4416 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.054604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-eb1b8a60-45c5-4b4e-a73c-dd79fb892f5c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:43:27.054710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-eb1b8a60-45c5-4b4e-a73c-dd79fb892f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:27.063528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1141/2283] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:43:27.064145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1143/2284] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:43:27.068698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b8a60-45c5-4b4e-a73c-dd79fb892f5c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.068848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6bbb7-cb43-40be-88a7-9f1083bf4416 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.077700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b8a60-45c5-4b4e-a73c-dd79fb892f5c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.083136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.083721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.083995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.084293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.084939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:43:27.085234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.085516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6bbb7-cb43-40be-88a7-9f1083bf4416 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.086809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.090806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab897da-4c6b-4b72-ae9c-9beb09290631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.091409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab897da-4c6b-4b72-ae9c-9beb09290631 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.092419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab897da-4c6b-4b72-ae9c-9beb09290631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.092712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab897da-4c6b-4b72-ae9c-9beb09290631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.093831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab897da-4c6b-4b72-ae9c-9beb09290631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.095767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab897da-4c6b-4b72-ae9c-9beb09290631 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.104240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1144/2285] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:43:27.109736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1142/2286] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:43:27.109874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.110610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab897da-4c6b-4b72-ae9c-9beb09290631 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.119375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313a5c-5a3a-4c78-8d18-d7051120bbfb admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.120005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab897da-4c6b-4b72-ae9c-9beb09290631 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.123984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc952e1c-31d7-4714-be7f-0be9628c9876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.124555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27250036-6a33-4666-b0b5-bd6f12b832c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.124699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc952e1c-31d7-4714-be7f-0be9628c9876 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.124868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc952e1c-31d7-4714-be7f-0be9628c9876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.125138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc952e1c-31d7-4714-be7f-0be9628c9876 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.125262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27250036-6a33-4666-b0b5-bd6f12b832c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.125437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27250036-6a33-4666-b0b5-bd6f12b832c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.126239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27250036-6a33-4666-b0b5-bd6f12b832c5 admin admin] PATH_INFO: `/v3/projects/07e61311e1d347a5960deb9582bf43c2/users/86bee9d5cd1047e28ed3f61385c886f5/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.126877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc952e1c-31d7-4714-be7f-0be9628c9876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.128808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc952e1c-31d7-4714-be7f-0be9628c9876 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.140477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27250036-6a33-4666-b0b5-bd6f12b832c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07e61311e1d347a5960deb9582bf43c2, user_id=86bee9d5cd1047e28ed3f61385c886f5, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.143773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27250036-6a33-4666-b0b5-bd6f12b832c5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.156790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1145/2287] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:43:27 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:43:27.157169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1143/2288] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:43:27 2022] PUT /identity/v3/projects/07e61311e1d347a5960deb9582bf43c2/users/86bee9d5cd1047e28ed3f61385c886f5/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:43:27.160965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc952e1c-31d7-4714-be7f-0be9628c9876 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.168038 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8707ed-20b7-4b82-94ac-8cc6788c410c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.168375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8707ed-20b7-4b82-94ac-8cc6788c410c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.168670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8707ed-20b7-4b82-94ac-8cc6788c410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.196632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-9b8707ed-20b7-4b82-94ac-8cc6788c410c None None] MFA Rules not processed for user `86bee9d5cd1047e28ed3f61385c886f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:43:27.197883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9b8707ed-20b7-4b82-94ac-8cc6788c410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:27.222683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:27.243911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc952e1c-31d7-4714-be7f-0be9628c9876 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.248793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdc9e4-b4b9-46c8-b500-1dc327fbefed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.249403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdc9e4-b4b9-46c8-b500-1dc327fbefed admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.249682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdc9e4-b4b9-46c8-b500-1dc327fbefed admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.249962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdc9e4-b4b9-46c8-b500-1dc327fbefed admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.251203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdc9e4-b4b9-46c8-b500-1dc327fbefed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.253470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bdc9e4-b4b9-46c8-b500-1dc327fbefed admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.266916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1144/2289] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:43:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:43:27.271689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8707ed-20b7-4b82-94ac-8cc6788c410c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.280150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1146/2290] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:43:27 2022] POST /identity/v3/users => generated 284 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:43:27.281336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdc9e4-b4b9-46c8-b500-1dc327fbefed admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.283104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8707ed-20b7-4b82-94ac-8cc6788c410c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.287882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6331b6c-c2ed-4f10-9e5d-718b7a224052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.288522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6331b6c-c2ed-4f10-9e5d-718b7a224052 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.288799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6331b6c-c2ed-4f10-9e5d-718b7a224052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.289065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6331b6c-c2ed-4f10-9e5d-718b7a224052 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.289700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6331b6c-c2ed-4f10-9e5d-718b7a224052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.291591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6331b6c-c2ed-4f10-9e5d-718b7a224052 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.292410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdc9e4-b4b9-46c8-b500-1dc327fbefed admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.297212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61357671-ce36-4887-9a24-0dd8a102b859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.298046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61357671-ce36-4887-9a24-0dd8a102b859 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.298333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61357671-ce36-4887-9a24-0dd8a102b859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.298633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61357671-ce36-4887-9a24-0dd8a102b859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.299451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61357671-ce36-4887-9a24-0dd8a102b859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.301090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61357671-ce36-4887-9a24-0dd8a102b859 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.311930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1147/2291] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:43:27.316709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61357671-ce36-4887-9a24-0dd8a102b859 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.321038 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1145/2292] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:43:27 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:43:27.324755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6331b6c-c2ed-4f10-9e5d-718b7a224052 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.330718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61357671-ce36-4887-9a24-0dd8a102b859 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.338408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:27.388466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6331b6c-c2ed-4f10-9e5d-718b7a224052 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.389096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47686-3128-4030-88e2-e91b11a1bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.389628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47686-3128-4030-88e2-e91b11a1bc51 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.389904 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47686-3128-4030-88e2-e91b11a1bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.390172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c47686-3128-4030-88e2-e91b11a1bc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.390806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c47686-3128-4030-88e2-e91b11a1bc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.393089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c47686-3128-4030-88e2-e91b11a1bc51 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.393995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238de52b-b408-41ad-8ef4-d4d09a715587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.394658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de52b-b408-41ad-8ef4-d4d09a715587 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.394939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de52b-b408-41ad-8ef4-d4d09a715587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.395239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de52b-b408-41ad-8ef4-d4d09a715587 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.395890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238de52b-b408-41ad-8ef4-d4d09a715587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.397873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238de52b-b408-41ad-8ef4-d4d09a715587 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.403055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1148/2293] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:43:27.407810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47686-3128-4030-88e2-e91b11a1bc51 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.418127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c47686-3128-4030-88e2-e91b11a1bc51 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.419584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1146/2294] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:43:27 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:43:27.423212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4abdcd-65d6-41cc-a9c1-a7cedb4bb768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.423408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238de52b-b408-41ad-8ef4-d4d09a715587 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.423857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4abdcd-65d6-41cc-a9c1-a7cedb4bb768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.424171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4abdcd-65d6-41cc-a9c1-a7cedb4bb768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.424448 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4abdcd-65d6-41cc-a9c1-a7cedb4bb768 admin admin] PATH_INFO: `/v3/projects/a847bf3d88a44535b3d52c471b4e2b58/users/e9ed37ef626e40689ec7181275a8d583/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.433151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238de52b-b408-41ad-8ef4-d4d09a715587 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.438077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5dcd6-5b44-4488-8642-030c9373647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.438712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5dcd6-5b44-4488-8642-030c9373647a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.438990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5dcd6-5b44-4488-8642-030c9373647a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.439255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5dcd6-5b44-4488-8642-030c9373647a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.439706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4abdcd-65d6-41cc-a9c1-a7cedb4bb768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a847bf3d88a44535b3d52c471b4e2b58, user_id=e9ed37ef626e40689ec7181275a8d583, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.439885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5dcd6-5b44-4488-8642-030c9373647a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.441474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5dcd6-5b44-4488-8642-030c9373647a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.442379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4abdcd-65d6-41cc-a9c1-a7cedb4bb768 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.451958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1147/2295] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:43:27.456543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1149/2296] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:43:27 2022] PUT /identity/v3/projects/a847bf3d88a44535b3d52c471b4e2b58/users/e9ed37ef626e40689ec7181275a8d583/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:43:27.457125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5dcd6-5b44-4488-8642-030c9373647a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.467375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5dcd6-5b44-4488-8642-030c9373647a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.468328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ce6e3-0e19-48a1-8b75-0a2af117efab None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.468690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ce6e3-0e19-48a1-8b75-0a2af117efab None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.469065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ce6e3-0e19-48a1-8b75-0a2af117efab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.497437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-087ce6e3-0e19-48a1-8b75-0a2af117efab None None] MFA Rules not processed for user `e9ed37ef626e40689ec7181275a8d583`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:43:27.498681 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-087ce6e3-0e19-48a1-8b75-0a2af117efab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:27.527512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e85b65-6e58-4fa8-8086-7ef6d361a425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.528187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e85b65-6e58-4fa8-8086-7ef6d361a425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.528469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e85b65-6e58-4fa8-8086-7ef6d361a425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.528740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e85b65-6e58-4fa8-8086-7ef6d361a425 admin admin] PATH_INFO: `/v3/projects/974ecdcb9c314071bca1d9629b73ac36/users/5bd7fe7cdbfe492cb38f9e640e090c14/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.544716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e85b65-6e58-4fa8-8086-7ef6d361a425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974ecdcb9c314071bca1d9629b73ac36, user_id=5bd7fe7cdbfe492cb38f9e640e090c14, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.549731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e85b65-6e58-4fa8-8086-7ef6d361a425 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.568180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1150/2297] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:43:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:43:27.569743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1148/2298] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:43:27 2022] PUT /identity/v3/projects/974ecdcb9c314071bca1d9629b73ac36/users/5bd7fe7cdbfe492cb38f9e640e090c14/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:43:27.573455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ce6e3-0e19-48a1-8b75-0a2af117efab None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.584363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ce6e3-0e19-48a1-8b75-0a2af117efab None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.641774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7e37c-dfd2-41e0-9f6c-47204ab9f50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.642405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7e37c-dfd2-41e0-9f6c-47204ab9f50d admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.642682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7e37c-dfd2-41e0-9f6c-47204ab9f50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.643000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7e37c-dfd2-41e0-9f6c-47204ab9f50d admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.643619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7e37c-dfd2-41e0-9f6c-47204ab9f50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.645251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7e37c-dfd2-41e0-9f6c-47204ab9f50d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.659177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1151/2299] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:43:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:43:27.660349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e85b65-6e58-4fa8-8086-7ef6d361a425 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:43:27.666969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:27.670777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e85b65-6e58-4fa8-8086-7ef6d361a425 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:43:27.676507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:43:27.677100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.677374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.677642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] PATH_INFO: `/v3/projects/974ecdcb9c314071bca1d9629b73ac36/users/5bd7fe7cdbfe492cb38f9e640e090c14/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.679510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974ecdcb9c314071bca1d9629b73ac36, user_id=5bd7fe7cdbfe492cb38f9e640e090c14, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:43:27.682141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:43:27.692979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1149/2300] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:43:27 2022] PUT /identity/v3/projects/974ecdcb9c314071bca1d9629b73ac36/users/5bd7fe7cdbfe492cb38f9e640e090c14/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:43:27.705068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61ab5-6271-46a2-8473-be294e19d7fe None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:43:27.705352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61ab5-6271-46a2-8473-be294e19d7fe None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:43:27.705622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61ab5-6271-46a2-8473-be294e19d7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:43:27.724558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:27.734226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d1c61ab5-6271-46a2-8473-be294e19d7fe None None] MFA Rules not processed for user `5bd7fe7cdbfe492cb38f9e640e090c14`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:43:27.735490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d1c61ab5-6271-46a2-8473-be294e19d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:43:27.803212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1152/2301] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:43:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:43:37.689330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=3,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:43:37.800355 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=30 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:24.508041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:24.509775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:24.569688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5e790-63ca-4d82-8b0f-af46480cca29 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:24.573286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa67f78-292f-4d0b-b19d-7321b26164c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:24.573880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67f78-292f-4d0b-b19d-7321b26164c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:24.574152 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67f78-292f-4d0b-b19d-7321b26164c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:24.574415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67f78-292f-4d0b-b19d-7321b26164c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:24.576123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-efa67f78-292f-4d0b-b19d-7321b26164c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:24.585927 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa67f78-292f-4d0b-b19d-7321b26164c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:24.587375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa67f78-292f-4d0b-b19d-7321b26164c4 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:24.595505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1150/2302] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:44:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:44:27.634073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61ab5-6271-46a2-8473-be294e19d7fe None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:27.695870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c61ab5-6271-46a2-8473-be294e19d7fe None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:27.700457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:27.701083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:27.701364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:27.701639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] PATH_INFO: `/v3/users/e9ed37ef626e40689ec7181275a8d583` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:27.703507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ed37ef626e40689ec7181275a8d583)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:27.705483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:27.750358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ed37ef626e40689ec7181275a8d583'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:27.761866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ed37ef626e40689ec7181275a8d583'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:27.762526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ed37ef626e40689ec7181275a8d583'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:27.770226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ed37ef626e40689ec7181275a8d583'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:27.777467 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ed37ef626e40689ec7181275a8d583'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:27.783622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1153/2303] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:44:27 2022] DELETE /identity/v3/users/e9ed37ef626e40689ec7181275a8d583 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:27.788170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723b707-a241-4841-b302-8575bcdcf8c5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:27.788643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723b707-a241-4841-b302-8575bcdcf8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:27.788743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723b707-a241-4841-b302-8575bcdcf8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:27.807357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7723b707-a241-4841-b302-8575bcdcf8c5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:27.808660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7723b707-a241-4841-b302-8575bcdcf8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:27.866706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1151/2304] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:44:27.877350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:27.878767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:27.937290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1e751-77e9-4f23-a43f-7d31180c0ff3 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:27.941011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199d439-b445-450d-8701-73fac073f912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:27.941607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199d439-b445-450d-8701-73fac073f912 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:27.941886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199d439-b445-450d-8701-73fac073f912 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:27.942158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199d439-b445-450d-8701-73fac073f912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:27.951052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199d439-b445-450d-8701-73fac073f912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:27.952568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199d439-b445-450d-8701-73fac073f912 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:27.961763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1154/2305] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:44:28.236672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723b707-a241-4841-b302-8575bcdcf8c5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:28.238212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7723b707-a241-4841-b302-8575bcdcf8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:28.250253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723b707-a241-4841-b302-8575bcdcf8c5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:28.255400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:28.256061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.256371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.256641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] PATH_INFO: `/v3/projects/a847bf3d88a44535b3d52c471b4e2b58` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.257910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a847bf3d88a44535b3d52c471b4e2b58)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:28.259719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:28.309130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a847bf3d88a44535b3d52c471b4e2b58', 'request_id': 'req-a6ee144c-1be4-442d-a023-62f42626107c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:28.317368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a847bf3d88a44535b3d52c471b4e2b58', 'request_id': 'req-a6ee144c-1be4-442d-a023-62f42626107c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:28.323895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a847bf3d88a44535b3d52c471b4e2b58', 'request_id': 'req-a6ee144c-1be4-442d-a023-62f42626107c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:28.334888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] The token cache is being invalidate because project a847bf3d88a44535b3d52c471b4e2b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:44:28.335179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:28.335813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-a6ee144c-1be4-442d-a023-62f42626107c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:28.337309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1152/2306] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:44:28 2022] DELETE /identity/v3/projects/a847bf3d88a44535b3d52c471b4e2b58 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:28.349318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6d11a-37e8-4f6a-bd1f-488779881649 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.349603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6d11a-37e8-4f6a-bd1f-488779881649 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.349875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6d11a-37e8-4f6a-bd1f-488779881649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.369045 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-82d6d11a-37e8-4f6a-bd1f-488779881649 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:28.370185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-82d6d11a-37e8-4f6a-bd1f-488779881649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:28.387257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:28.431724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1155/2307] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:44:28.440574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb021d-c8f5-40fd-a8d9-b854f13f6f85 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.440858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb021d-c8f5-40fd-a8d9-b854f13f6f85 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.441129 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb021d-c8f5-40fd-a8d9-b854f13f6f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.460728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b4eb021d-c8f5-40fd-a8d9-b854f13f6f85 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:28.461907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b4eb021d-c8f5-40fd-a8d9-b854f13f6f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:28.469655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1153/2308] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:44:28.473723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6d11a-37e8-4f6a-bd1f-488779881649 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:28.483307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6d11a-37e8-4f6a-bd1f-488779881649 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:28.487940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:28.488574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.488854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.489129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.489775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:44:28.490078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:28.491657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:28.503831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1156/2309] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:44:28.508351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb021d-c8f5-40fd-a8d9-b854f13f6f85 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:28.518085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb021d-c8f5-40fd-a8d9-b854f13f6f85 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:28.522740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60cf98-7914-4e7f-aeac-261cacc0daa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:28.523331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60cf98-7914-4e7f-aeac-261cacc0daa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.523616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60cf98-7914-4e7f-aeac-261cacc0daa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.523888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60cf98-7914-4e7f-aeac-261cacc0daa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.524546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60cf98-7914-4e7f-aeac-261cacc0daa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:28.526461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60cf98-7914-4e7f-aeac-261cacc0daa8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:28.546555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:28.555489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1154/2310] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:44:28 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:44:28.559472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:28.570123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec019d2-4795-4bfc-adec-af18a522daef admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:28.625817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aea9e8-55bc-44d9-86c6-72faf5b56d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:28.626338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aea9e8-55bc-44d9-86c6-72faf5b56d6e admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.626618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aea9e8-55bc-44d9-86c6-72faf5b56d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.626891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aea9e8-55bc-44d9-86c6-72faf5b56d6e admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.627513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aea9e8-55bc-44d9-86c6-72faf5b56d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:28.629454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aea9e8-55bc-44d9-86c6-72faf5b56d6e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:28.650108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1157/2311] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:44:28 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:44:28.654179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60cf98-7914-4e7f-aeac-261cacc0daa8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:28.664403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60cf98-7914-4e7f-aeac-261cacc0daa8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:28.669205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00262883-4148-44db-98e3-5b3f0550aa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:28.669804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00262883-4148-44db-98e3-5b3f0550aa04 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.670075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00262883-4148-44db-98e3-5b3f0550aa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.670337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00262883-4148-44db-98e3-5b3f0550aa04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.671538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00262883-4148-44db-98e3-5b3f0550aa04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:28.673291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00262883-4148-44db-98e3-5b3f0550aa04 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:28.683119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1155/2312] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:44:28.687792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aea9e8-55bc-44d9-86c6-72faf5b56d6e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:28.697625 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aea9e8-55bc-44d9-86c6-72faf5b56d6e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:28.702329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6139c-6302-4556-b405-f5ff3282f813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:28.702918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6139c-6302-4556-b405-f5ff3282f813 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.703196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6139c-6302-4556-b405-f5ff3282f813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.703466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6139c-6302-4556-b405-f5ff3282f813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.704162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6139c-6302-4556-b405-f5ff3282f813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:28.705899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6139c-6302-4556-b405-f5ff3282f813 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:28.716379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1158/2313] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:44:28.720290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00262883-4148-44db-98e3-5b3f0550aa04 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:28.730244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00262883-4148-44db-98e3-5b3f0550aa04 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:28.735506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:28.736170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.736448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.736726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] PATH_INFO: `/v3/projects/61eb54edc35b4c90ba6d116eb8994459/users/15ce376bf5784d529fccb8f9d7716d13/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.751516 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=61eb54edc35b4c90ba6d116eb8994459, user_id=15ce376bf5784d529fccb8f9d7716d13, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:28.754178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:28.766474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1156/2314] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:44:28 2022] PUT /identity/v3/projects/61eb54edc35b4c90ba6d116eb8994459/users/15ce376bf5784d529fccb8f9d7716d13/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:28.771491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebbee0-3f3a-41c6-b7da-357daee4cb8e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.771776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebbee0-3f3a-41c6-b7da-357daee4cb8e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.772076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ebbee0-3f3a-41c6-b7da-357daee4cb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.790721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-28ebbee0-3f3a-41c6-b7da-357daee4cb8e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:28.791923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-28ebbee0-3f3a-41c6-b7da-357daee4cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:28.849231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1159/2315] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:44:28.858611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:28.860981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:28.916447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3e6fb-967f-4109-ba0e-f4d007f6f1cf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:28.920012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94710277-5998-4258-a6b9-f475dbb85ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:28.921140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94710277-5998-4258-a6b9-f475dbb85ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:28.921434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94710277-5998-4258-a6b9-f475dbb85ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:28.921703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94710277-5998-4258-a6b9-f475dbb85ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:28.930453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94710277-5998-4258-a6b9-f475dbb85ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:28.931851 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94710277-5998-4258-a6b9-f475dbb85ce6 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:28.942188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1157/2316] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:44:29.569364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce637f-50f4-453e-a1b4-16cfa0b8f296 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:29.569677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce637f-50f4-453e-a1b4-16cfa0b8f296 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:29.569950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce637f-50f4-453e-a1b4-16cfa0b8f296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:29.599503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-57ce637f-50f4-453e-a1b4-16cfa0b8f296 None None] MFA Rules not processed for user `15ce376bf5784d529fccb8f9d7716d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:29.600907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-57ce637f-50f4-453e-a1b4-16cfa0b8f296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:29.618092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:29.671940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1160/2317] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:29 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:44:29.676650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94710277-5998-4258-a6b9-f475dbb85ce6 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:29.687405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94710277-5998-4258-a6b9-f475dbb85ce6 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:29.692541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29c2a7-d560-47a2-81ba-92629ce2058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:29.693134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c2a7-d560-47a2-81ba-92629ce2058b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:29.693404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c2a7-d560-47a2-81ba-92629ce2058b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:29.693670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c2a7-d560-47a2-81ba-92629ce2058b admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:29.694298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29c2a7-d560-47a2-81ba-92629ce2058b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:29.696191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29c2a7-d560-47a2-81ba-92629ce2058b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:29.724037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1158/2318] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:44:29 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:44:29.728391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce637f-50f4-453e-a1b4-16cfa0b8f296 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:29.739884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce637f-50f4-453e-a1b4-16cfa0b8f296 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:29.796222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8762e-3133-4561-bd27-13df9196ce88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:29.796831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8762e-3133-4561-bd27-13df9196ce88 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:29.797115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8762e-3133-4561-bd27-13df9196ce88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:29.797385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8762e-3133-4561-bd27-13df9196ce88 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:29.798022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8762e-3133-4561-bd27-13df9196ce88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:29.799937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8762e-3133-4561-bd27-13df9196ce88 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:29.822084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1161/2319] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:44:29 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:44:29.826410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29c2a7-d560-47a2-81ba-92629ce2058b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:29.837542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29c2a7-d560-47a2-81ba-92629ce2058b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:29.842069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2fb1c-05b6-4f74-9a55-b6b194b04566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:29.842693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2fb1c-05b6-4f74-9a55-b6b194b04566 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:29.842965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2fb1c-05b6-4f74-9a55-b6b194b04566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:29.843231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2fb1c-05b6-4f74-9a55-b6b194b04566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:29.843864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2fb1c-05b6-4f74-9a55-b6b194b04566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:29.845466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2fb1c-05b6-4f74-9a55-b6b194b04566 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:29.855611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1159/2320] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:44:29.859621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8762e-3133-4561-bd27-13df9196ce88 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:29.869587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8762e-3133-4561-bd27-13df9196ce88 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:29.874708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470ccf2-2d65-4eb7-a204-060434708dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:29.875425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470ccf2-2d65-4eb7-a204-060434708dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:29.875705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470ccf2-2d65-4eb7-a204-060434708dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:29.875977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470ccf2-2d65-4eb7-a204-060434708dfe admin admin] PATH_INFO: `/v3/projects/bc03f0e70db441c5ae1751026f4d76f8/users/438d1d8e35384cfabb6dd429da9f4c92/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:29.891084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470ccf2-2d65-4eb7-a204-060434708dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc03f0e70db441c5ae1751026f4d76f8, user_id=438d1d8e35384cfabb6dd429da9f4c92, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:29.893843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470ccf2-2d65-4eb7-a204-060434708dfe admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:29.907083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1162/2321] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:44:29 2022] PUT /identity/v3/projects/bc03f0e70db441c5ae1751026f4d76f8/users/438d1d8e35384cfabb6dd429da9f4c92/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:29.910978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2fb1c-05b6-4f74-9a55-b6b194b04566 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:29.921204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2fb1c-05b6-4f74-9a55-b6b194b04566 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:29.977007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c343d26-ab94-46e7-b580-0943a7b44c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:29.977535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c343d26-ab94-46e7-b580-0943a7b44c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:29.977815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c343d26-ab94-46e7-b580-0943a7b44c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:29.978087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c343d26-ab94-46e7-b580-0943a7b44c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:29.978722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c343d26-ab94-46e7-b580-0943a7b44c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:29.980347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c343d26-ab94-46e7-b580-0943a7b44c11 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:29.990021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1160/2322] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:44:29.994095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470ccf2-2d65-4eb7-a204-060434708dfe admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:30.004792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470ccf2-2d65-4eb7-a204-060434708dfe admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:30.009311 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdd000-98a7-4b8e-85b4-5d96dbd15eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:30.009941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdd000-98a7-4b8e-85b4-5d96dbd15eff admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:30.010227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdd000-98a7-4b8e-85b4-5d96dbd15eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:30.010478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdd000-98a7-4b8e-85b4-5d96dbd15eff admin admin] PATH_INFO: `/v3/projects/bc03f0e70db441c5ae1751026f4d76f8/users/438d1d8e35384cfabb6dd429da9f4c92/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:30.012363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdd000-98a7-4b8e-85b4-5d96dbd15eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc03f0e70db441c5ae1751026f4d76f8, user_id=438d1d8e35384cfabb6dd429da9f4c92, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:30.015736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdd000-98a7-4b8e-85b4-5d96dbd15eff admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:30.025703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1163/2323] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:44:29 2022] PUT /identity/v3/projects/bc03f0e70db441c5ae1751026f4d76f8/users/438d1d8e35384cfabb6dd429da9f4c92/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:30.648591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1615586-2240-4c0e-865a-c55afc30b271 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:30.648877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1615586-2240-4c0e-865a-c55afc30b271 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:30.649150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1615586-2240-4c0e-865a-c55afc30b271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:30.679903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f1615586-2240-4c0e-865a-c55afc30b271 None None] MFA Rules not processed for user `438d1d8e35384cfabb6dd429da9f4c92`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:30.681170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f1615586-2240-4c0e-865a-c55afc30b271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:30.698421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:30.755616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1161/2324] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:44:31.764444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdd000-98a7-4b8e-85b4-5d96dbd15eff admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:31.828143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdd000-98a7-4b8e-85b4-5d96dbd15eff admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:31.832637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:31.833277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:31.833550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:31.833818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] PATH_INFO: `/v3/users/15ce376bf5784d529fccb8f9d7716d13` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:31.834938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ce376bf5784d529fccb8f9d7716d13)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:31.836879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:31.848771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:31.883780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ce376bf5784d529fccb8f9d7716d13'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:31.895614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ce376bf5784d529fccb8f9d7716d13'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:31.896337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ce376bf5784d529fccb8f9d7716d13'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:31.903082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ce376bf5784d529fccb8f9d7716d13'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:31.910659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ce376bf5784d529fccb8f9d7716d13'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:31.916719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1164/2325] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:44:31 2022] DELETE /identity/v3/users/15ce376bf5784d529fccb8f9d7716d13 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:32.087699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1615586-2240-4c0e-865a-c55afc30b271 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:32.099226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1615586-2240-4c0e-865a-c55afc30b271 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:32.160417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:32.161082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:32.161360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:32.161633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] PATH_INFO: `/v3/projects/61eb54edc35b4c90ba6d116eb8994459` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:32.162748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61eb54edc35b4c90ba6d116eb8994459)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:32.164725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:32.211593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61eb54edc35b4c90ba6d116eb8994459', 'request_id': 'req-2143982d-1468-4b6d-961c-9cfe2a3d28a2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.218896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61eb54edc35b4c90ba6d116eb8994459', 'request_id': 'req-2143982d-1468-4b6d-961c-9cfe2a3d28a2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.225045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61eb54edc35b4c90ba6d116eb8994459', 'request_id': 'req-2143982d-1468-4b6d-961c-9cfe2a3d28a2'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.236596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] The token cache is being invalidate because project 61eb54edc35b4c90ba6d116eb8994459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:44:32.236878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.237507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.238970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1162/2326] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:44:32 2022] DELETE /identity/v3/projects/61eb54edc35b4c90ba6d116eb8994459 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:32.242918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:32.244373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:32.258091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dc898-2987-4a40-baba-dea286b009cf admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:32.313969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:32.314598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:32.315310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:32.315310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] PATH_INFO: `/v3/users/438d1d8e35384cfabb6dd429da9f4c92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:32.316276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438d1d8e35384cfabb6dd429da9f4c92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:32.318195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:32.365357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438d1d8e35384cfabb6dd429da9f4c92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.378103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438d1d8e35384cfabb6dd429da9f4c92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.378783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438d1d8e35384cfabb6dd429da9f4c92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.385615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438d1d8e35384cfabb6dd429da9f4c92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.393582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-064b9255-ad47-4df1-975f-4d544d811ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438d1d8e35384cfabb6dd429da9f4c92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.399797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1165/2327] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:44:32 2022] DELETE /identity/v3/users/438d1d8e35384cfabb6dd429da9f4c92 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:32.570384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:32.571802 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:32.585154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143982d-1468-4b6d-961c-9cfe2a3d28a2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:32.627631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:32.643981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:32.644644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:32.644915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:32.645181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] PATH_INFO: `/v3/projects/bc03f0e70db441c5ae1751026f4d76f8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:32.646276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc03f0e70db441c5ae1751026f4d76f8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:32.648239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:32.693252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc03f0e70db441c5ae1751026f4d76f8', 'request_id': 'req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.700179 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc03f0e70db441c5ae1751026f4d76f8', 'request_id': 'req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.707133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc03f0e70db441c5ae1751026f4d76f8', 'request_id': 'req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.717133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] The token cache is being invalidate because project bc03f0e70db441c5ae1751026f4d76f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:44:32.717418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.719093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f1e2e0fa-6c0f-42f1-8b85-cf09e188d11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:44:32.720607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1163/2328] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:44:32 2022] DELETE /identity/v3/projects/bc03f0e70db441c5ae1751026f4d76f8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:32.725513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a6736-003c-44c1-9259-d40a06cde15b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:32.725794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a6736-003c-44c1-9259-d40a06cde15b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:32.726065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a6736-003c-44c1-9259-d40a06cde15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:32.745110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-6e9a6736-003c-44c1-9259-d40a06cde15b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:32.746295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6e9a6736-003c-44c1-9259-d40a06cde15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:32.750474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:32.805532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1166/2329] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:44:32.814203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315f457-e9dc-4f7a-af9a-314538b6c069 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:32.814483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315f457-e9dc-4f7a-af9a-314538b6c069 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:32.814753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315f457-e9dc-4f7a-af9a-314538b6c069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:32.834479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8315f457-e9dc-4f7a-af9a-314538b6c069 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:32.835660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8315f457-e9dc-4f7a-af9a-314538b6c069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:32.843448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1164/2330] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:44:32.847472 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a6736-003c-44c1-9259-d40a06cde15b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:32.857208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a6736-003c-44c1-9259-d40a06cde15b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:32.861988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:32.862589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:32.862869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:32.863140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:32.863785 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:44:32.864087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:32.865699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:32.877621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1167/2331] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:44:32.881797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315f457-e9dc-4f7a-af9a-314538b6c069 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:32.891523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315f457-e9dc-4f7a-af9a-314538b6c069 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:32.896156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413d74-b478-4edb-8cdc-e3f88ec00dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:32.896763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413d74-b478-4edb-8cdc-e3f88ec00dbd admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:32.897039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413d74-b478-4edb-8cdc-e3f88ec00dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:32.897316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4413d74-b478-4edb-8cdc-e3f88ec00dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:32.898996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4413d74-b478-4edb-8cdc-e3f88ec00dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:32.900673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4413d74-b478-4edb-8cdc-e3f88ec00dbd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:32.930321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1165/2332] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:44:32 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:44:32.934515 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:32.945517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1182c-2783-448f-9fd4-27e09522f0e7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.002534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538b813-898a-49c3-94b9-13d4ddb756d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.003128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538b813-898a-49c3-94b9-13d4ddb756d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.003407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538b813-898a-49c3-94b9-13d4ddb756d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.003679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538b813-898a-49c3-94b9-13d4ddb756d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.004344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538b813-898a-49c3-94b9-13d4ddb756d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.006254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538b813-898a-49c3-94b9-13d4ddb756d3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.028272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1168/2333] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:44:32 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:44:33.032685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413d74-b478-4edb-8cdc-e3f88ec00dbd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.043102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4413d74-b478-4edb-8cdc-e3f88ec00dbd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.047971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b9a93-be53-40a1-a0ae-5b22bf2c7fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.048602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b9a93-be53-40a1-a0ae-5b22bf2c7fba admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.048878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b9a93-be53-40a1-a0ae-5b22bf2c7fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.049147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b9a93-be53-40a1-a0ae-5b22bf2c7fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.049789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b9a93-be53-40a1-a0ae-5b22bf2c7fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.051367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82b9a93-be53-40a1-a0ae-5b22bf2c7fba admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.062430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1166/2334] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:44:33.066631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538b813-898a-49c3-94b9-13d4ddb756d3 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.076429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538b813-898a-49c3-94b9-13d4ddb756d3 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.081858 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5bb16-470e-4203-bd30-3296f0e88d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.082437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5bb16-470e-4203-bd30-3296f0e88d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.082710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5bb16-470e-4203-bd30-3296f0e88d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.082977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5bb16-470e-4203-bd30-3296f0e88d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.083609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5bb16-470e-4203-bd30-3296f0e88d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.085237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5bb16-470e-4203-bd30-3296f0e88d45 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.095541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1169/2335] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:44:33.099601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b9a93-be53-40a1-a0ae-5b22bf2c7fba admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.109735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b9a93-be53-40a1-a0ae-5b22bf2c7fba admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.114367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1832ad6-583b-4847-99f4-cfbfd3a5c899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.115006 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832ad6-583b-4847-99f4-cfbfd3a5c899 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.115287 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832ad6-583b-4847-99f4-cfbfd3a5c899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.115562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832ad6-583b-4847-99f4-cfbfd3a5c899 admin admin] PATH_INFO: `/v3/projects/ee001eaf79d848e99282761e1b637b0a/users/75d95fc1a9f04afba1173548e06a219c/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.130788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1832ad6-583b-4847-99f4-cfbfd3a5c899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee001eaf79d848e99282761e1b637b0a, user_id=75d95fc1a9f04afba1173548e06a219c, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.133506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1832ad6-583b-4847-99f4-cfbfd3a5c899 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.146330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1167/2336] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:44:33 2022] PUT /identity/v3/projects/ee001eaf79d848e99282761e1b637b0a/users/75d95fc1a9f04afba1173548e06a219c/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:33.156980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b1af-ebbb-41e9-a0c0-e70b67ccc425 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.157413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b1af-ebbb-41e9-a0c0-e70b67ccc425 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.157689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404b1af-ebbb-41e9-a0c0-e70b67ccc425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.185209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3404b1af-ebbb-41e9-a0c0-e70b67ccc425 None None] MFA Rules not processed for user `75d95fc1a9f04afba1173548e06a219c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:33.187828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3404b1af-ebbb-41e9-a0c0-e70b67ccc425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:33.252721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1170/2337] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:44:33.257169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1832ad6-583b-4847-99f4-cfbfd3a5c899 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.324737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1832ad6-583b-4847-99f4-cfbfd3a5c899 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.330343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7160-6393-4056-b77d-9dd2fa5d6ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.330937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7160-6393-4056-b77d-9dd2fa5d6ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.331209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7160-6393-4056-b77d-9dd2fa5d6ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.331475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7160-6393-4056-b77d-9dd2fa5d6ae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.332117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb7160-6393-4056-b77d-9dd2fa5d6ae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.333986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb7160-6393-4056-b77d-9dd2fa5d6ae0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.349494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:33.363319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1168/2338] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:44:33 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:44:33.366901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404b1af-ebbb-41e9-a0c0-e70b67ccc425 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.373907 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:33.381885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404b1af-ebbb-41e9-a0c0-e70b67ccc425 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.440561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e385ee9-a6dd-408b-94ba-cb539c68df55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.441588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e385ee9-a6dd-408b-94ba-cb539c68df55 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.441588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e385ee9-a6dd-408b-94ba-cb539c68df55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.441588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e385ee9-a6dd-408b-94ba-cb539c68df55 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.442338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e385ee9-a6dd-408b-94ba-cb539c68df55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.444317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e385ee9-a6dd-408b-94ba-cb539c68df55 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.464892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1171/2339] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:44:33 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:44:33.469566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7160-6393-4056-b77d-9dd2fa5d6ae0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.479840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7160-6393-4056-b77d-9dd2fa5d6ae0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.484830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d723ea-5a99-4236-826e-a4f8b78fc566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.485429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d723ea-5a99-4236-826e-a4f8b78fc566 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.485710 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d723ea-5a99-4236-826e-a4f8b78fc566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.485985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d723ea-5a99-4236-826e-a4f8b78fc566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.486651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d723ea-5a99-4236-826e-a4f8b78fc566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.488271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d723ea-5a99-4236-826e-a4f8b78fc566 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.498790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1169/2340] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:44:33.502922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e385ee9-a6dd-408b-94ba-cb539c68df55 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.512853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e385ee9-a6dd-408b-94ba-cb539c68df55 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.518222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6fa75-8898-4cb0-85b2-93a52df54982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.518888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fa75-8898-4cb0-85b2-93a52df54982 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.519168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fa75-8898-4cb0-85b2-93a52df54982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.519438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fa75-8898-4cb0-85b2-93a52df54982 admin admin] PATH_INFO: `/v3/projects/a9b7d28d0e9a400a9658b496c2ebccd3/users/4a09a16a00354a71bebce49b159b8268/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.534542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6fa75-8898-4cb0-85b2-93a52df54982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b7d28d0e9a400a9658b496c2ebccd3, user_id=4a09a16a00354a71bebce49b159b8268, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.537219 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6fa75-8898-4cb0-85b2-93a52df54982 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.549289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1172/2341] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:44:33 2022] PUT /identity/v3/projects/a9b7d28d0e9a400a9658b496c2ebccd3/users/4a09a16a00354a71bebce49b159b8268/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:44:33.553350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d723ea-5a99-4236-826e-a4f8b78fc566 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.564170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d723ea-5a99-4236-826e-a4f8b78fc566 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.619357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23bf36-9a4d-4a47-befa-42705b6e2a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.619950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23bf36-9a4d-4a47-befa-42705b6e2a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.620332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23bf36-9a4d-4a47-befa-42705b6e2a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.620600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23bf36-9a4d-4a47-befa-42705b6e2a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.621225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23bf36-9a4d-4a47-befa-42705b6e2a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.622809 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23bf36-9a4d-4a47-befa-42705b6e2a42 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.632858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1170/2342] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:44:33.637262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6fa75-8898-4cb0-85b2-93a52df54982 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:44:33.647937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6fa75-8898-4cb0-85b2-93a52df54982 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:44:33.652462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:44:33.653094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.653369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.653633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] PATH_INFO: `/v3/projects/a9b7d28d0e9a400a9658b496c2ebccd3/users/4a09a16a00354a71bebce49b159b8268/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.655687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b7d28d0e9a400a9658b496c2ebccd3, user_id=4a09a16a00354a71bebce49b159b8268, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:44:33.658327 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:44:33.668270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1173/2343] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:44:33 2022] PUT /identity/v3/projects/a9b7d28d0e9a400a9658b496c2ebccd3/users/4a09a16a00354a71bebce49b159b8268/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:44:33.679763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8339ce4-bd58-45fa-8b15-7a80388738c9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:44:33.680047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8339ce4-bd58-45fa-8b15-7a80388738c9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:44:33.680351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8339ce4-bd58-45fa-8b15-7a80388738c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:44:33.709478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f8339ce4-bd58-45fa-8b15-7a80388738c9 None None] MFA Rules not processed for user `4a09a16a00354a71bebce49b159b8268`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:44:33.710667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f8339ce4-bd58-45fa-8b15-7a80388738c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:44:33.778548 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1171/2344] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:44:43.664610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=42,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:44:43.775626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=70 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:45:19.137248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:19.139111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:19.196539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f9eaf-2387-4ddb-9676-6e43bd8f16bc admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:19.200180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc1930-6ed0-4569-8122-983683b84f59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:19.200938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc1930-6ed0-4569-8122-983683b84f59 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:19.201216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc1930-6ed0-4569-8122-983683b84f59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:19.201482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc1930-6ed0-4569-8122-983683b84f59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:19.203233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b4bc1930-6ed0-4569-8122-983683b84f59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application [None req-b4bc1930-6ed0-4569-8122-983683b84f59 service cinder] Could not find project: a847bf3d88a44535b3d52c471b4e2b58.: keystone.exception.ProjectNotFound: Could not find project: a847bf3d88a44535b3d52c471b4e2b58. Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 11 16:45:19.215732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:45:19.217067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:45:19.218488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application with Lock( Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:45:19.220469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:45:19.221731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a847bf3d88a44535b3d52c471b4e2b58. Aug 11 16:45:19.221731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: ERROR keystone.server.flask.application Aug 11 16:45:19.221731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1174/2345] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:45:19 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 80 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 11 16:45:19.260115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8339ce4-bd58-45fa-8b15-7a80388738c9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:19.270935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8339ce4-bd58-45fa-8b15-7a80388738c9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:19.274558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:19.275154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:19.275430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:19.275696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:19.277428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:19.342446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:19.343859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:19.353734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1172/2346] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:45:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:45:19.537668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc1930-6ed0-4569-8122-983683b84f59 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:19.546791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc1930-6ed0-4569-8122-983683b84f59 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:19.551080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a5cb9-5e20-4187-a461-4d78a8a4ebfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:19.551670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5cb9-5e20-4187-a461-4d78a8a4ebfb service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:19.551945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5cb9-5e20-4187-a461-4d78a8a4ebfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:19.552251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5cb9-5e20-4187-a461-4d78a8a4ebfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:19.561342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a5cb9-5e20-4187-a461-4d78a8a4ebfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:19.563660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a5cb9-5e20-4187-a461-4d78a8a4ebfb service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:19.573754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1175/2347] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:45:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:45:21.138284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:21.140941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:21.235782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a5cb9-5e20-4187-a461-4d78a8a4ebfb service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:21.263608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f3090-965f-4240-a37f-81d587670bfe service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:21.271689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563203d-8d08-4e0e-9bb2-2fc3db6b25fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:21.272352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563203d-8d08-4e0e-9bb2-2fc3db6b25fc service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:21.272634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563203d-8d08-4e0e-9bb2-2fc3db6b25fc service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:21.272910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563203d-8d08-4e0e-9bb2-2fc3db6b25fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:21.274932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d563203d-8d08-4e0e-9bb2-2fc3db6b25fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:21.293144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563203d-8d08-4e0e-9bb2-2fc3db6b25fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:21.294857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563203d-8d08-4e0e-9bb2-2fc3db6b25fc service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:21.326978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1173/2348] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:45:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:45:21.363663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a5cb9-5e20-4187-a461-4d78a8a4ebfb service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:21.372347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:21.373222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:21.373540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:21.373848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] PATH_INFO: `/v3/users/75d95fc1a9f04afba1173548e06a219c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:21.375188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d95fc1a9f04afba1173548e06a219c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:21.380304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:21.458446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d95fc1a9f04afba1173548e06a219c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:21.471954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d95fc1a9f04afba1173548e06a219c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:21.472802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d95fc1a9f04afba1173548e06a219c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:21.479827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d95fc1a9f04afba1173548e06a219c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:21.487885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d95fc1a9f04afba1173548e06a219c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:21.494040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1176/2349] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:45:21 2022] DELETE /identity/v3/users/75d95fc1a9f04afba1173548e06a219c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:45:21.498496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2858-9961-4ff6-b9f4-159bdfac27f2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:21.498803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2858-9961-4ff6-b9f4-159bdfac27f2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:21.499079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c2858-9961-4ff6-b9f4-159bdfac27f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:21.518373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-108c2858-9961-4ff6-b9f4-159bdfac27f2 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:45:21.519574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-108c2858-9961-4ff6-b9f4-159bdfac27f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:21.578056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1174/2350] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:45:21.589558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:21.591041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:21.607781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:45:21.661353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5478e-9f11-43d3-b784-6ab24ef6fdcc admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:21.664976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:21.665580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:21.665857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:21.666123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:21.675921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:21.677371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:21.686702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1177/2351] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:45:21.998948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c2858-9961-4ff6-b9f4-159bdfac27f2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:22.001064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-108c2858-9961-4ff6-b9f4-159bdfac27f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:22.013790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c2858-9961-4ff6-b9f4-159bdfac27f2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:22.018456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:22.019290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:22.019568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:22.019837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:22.021613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:22.042593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:45:22.129089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:22.130542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:22.144416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1175/2352] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:45:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:45:22.255440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:22.257087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:22.272276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26079f3a-699d-40f4-8c56-552972c1ceb5 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:22.277029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:22.277666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:22.277947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:22.278220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] PATH_INFO: `/v3/projects/ee001eaf79d848e99282761e1b637b0a` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:22.279426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee001eaf79d848e99282761e1b637b0a)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:22.281369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:22.341136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee001eaf79d848e99282761e1b637b0a', 'request_id': 'req-e38cc59b-f28f-4cca-b99f-b0aa759c4003'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.348589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee001eaf79d848e99282761e1b637b0a', 'request_id': 'req-e38cc59b-f28f-4cca-b99f-b0aa759c4003'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.354208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee001eaf79d848e99282761e1b637b0a', 'request_id': 'req-e38cc59b-f28f-4cca-b99f-b0aa759c4003'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.366604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] The token cache is being invalidate because project ee001eaf79d848e99282761e1b637b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:45:22.366892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.367507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.369021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1178/2353] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:45:22 2022] DELETE /identity/v3/projects/ee001eaf79d848e99282761e1b637b0a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:45:22.373016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:22.383703 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ff983-bea5-438d-a85d-7a560d803060 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:22.439629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:22.440293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:22.440575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:22.440849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] PATH_INFO: `/v3/users/4a09a16a00354a71bebce49b159b8268` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:22.441968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a09a16a00354a71bebce49b159b8268)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:22.443881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:22.489460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a09a16a00354a71bebce49b159b8268'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.497142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a09a16a00354a71bebce49b159b8268'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.497756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a09a16a00354a71bebce49b159b8268'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.510653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a09a16a00354a71bebce49b159b8268'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.511860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-84f72e25-a48c-4cf9-84ee-57d8f2678310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a09a16a00354a71bebce49b159b8268'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.524884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1176/2354] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:45:22 2022] DELETE /identity/v3/users/4a09a16a00354a71bebce49b159b8268 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:45:22.792951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:22.794502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:22.808536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38cc59b-f28f-4cca-b99f-b0aa759c4003 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:22.870344 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:22.870886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:22.871162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:22.871429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] PATH_INFO: `/v3/projects/a9b7d28d0e9a400a9658b496c2ebccd3` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:22.872895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b7d28d0e9a400a9658b496c2ebccd3)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:22.874791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:22.916816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:45:22.924148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b7d28d0e9a400a9658b496c2ebccd3', 'request_id': 'req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.931796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b7d28d0e9a400a9658b496c2ebccd3', 'request_id': 'req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.938639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b7d28d0e9a400a9658b496c2ebccd3', 'request_id': 'req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.951322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] The token cache is being invalidate because project a9b7d28d0e9a400a9658b496c2ebccd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:45:22.951607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.952242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-e33bf63a-9046-4ce0-b063-78ebd3ce44f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:45:22.954445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1179/2355] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:45:22 2022] DELETE /identity/v3/projects/a9b7d28d0e9a400a9658b496c2ebccd3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:45:22.960137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5de0-1d9e-40ef-8944-d2e86bd46efa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:22.960423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5de0-1d9e-40ef-8944-d2e86bd46efa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:22.960694 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d5de0-1d9e-40ef-8944-d2e86bd46efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:22.980134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d47d5de0-1d9e-40ef-8944-d2e86bd46efa None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:45:22.981373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d47d5de0-1d9e-40ef-8944-d2e86bd46efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:23.000931 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:45:23.043238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1177/2356] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:45:23.051536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902dd20-8437-416f-86d4-262d55fb7c2b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.051867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902dd20-8437-416f-86d4-262d55fb7c2b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.052196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902dd20-8437-416f-86d4-262d55fb7c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.073353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e902dd20-8437-416f-86d4-262d55fb7c2b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:45:23.074551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e902dd20-8437-416f-86d4-262d55fb7c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:23.082620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1180/2357] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:45:23.086847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d5de0-1d9e-40ef-8944-d2e86bd46efa None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:23.096637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d5de0-1d9e-40ef-8944-d2e86bd46efa None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:23.101185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:23.101884 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.102162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.102445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.103067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:45:23.103365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:23.105016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:23.116989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1178/2358] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:45:23.121136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902dd20-8437-416f-86d4-262d55fb7c2b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:23.130785 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902dd20-8437-416f-86d4-262d55fb7c2b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:23.135372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f0aa8-2858-4e5e-a54f-76bd28a1de18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:23.135956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0aa8-2858-4e5e-a54f-76bd28a1de18 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.136269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0aa8-2858-4e5e-a54f-76bd28a1de18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.136539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0aa8-2858-4e5e-a54f-76bd28a1de18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.137164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f0aa8-2858-4e5e-a54f-76bd28a1de18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:23.139020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f0aa8-2858-4e5e-a54f-76bd28a1de18 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:23.164588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1181/2359] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:45:23 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:45:23.168695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:23.179515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a09670-2a34-407b-bb56-e459de77fb04 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:23.238224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcd6e0-0eda-4a1c-b964-4ca710f60d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:23.238800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd6e0-0eda-4a1c-b964-4ca710f60d29 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.239102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd6e0-0eda-4a1c-b964-4ca710f60d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.239373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcd6e0-0eda-4a1c-b964-4ca710f60d29 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.239999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcd6e0-0eda-4a1c-b964-4ca710f60d29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:23.241961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcd6e0-0eda-4a1c-b964-4ca710f60d29 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:23.263831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1179/2360] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:45:23 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:45:23.268302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f0aa8-2858-4e5e-a54f-76bd28a1de18 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:23.278865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f0aa8-2858-4e5e-a54f-76bd28a1de18 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:23.284398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b9925-0c4e-4329-af6c-867abdb8a88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:23.285020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b9925-0c4e-4329-af6c-867abdb8a88d admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.285301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b9925-0c4e-4329-af6c-867abdb8a88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.285575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b9925-0c4e-4329-af6c-867abdb8a88d admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.286232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b9925-0c4e-4329-af6c-867abdb8a88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:23.287838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b9925-0c4e-4329-af6c-867abdb8a88d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:23.298074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1182/2361] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:45:23.302140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcd6e0-0eda-4a1c-b964-4ca710f60d29 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:23.312419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcd6e0-0eda-4a1c-b964-4ca710f60d29 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:23.320942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fb93a-093b-4cec-87ee-0b62e4940932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:23.320942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb93a-093b-4cec-87ee-0b62e4940932 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.320942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb93a-093b-4cec-87ee-0b62e4940932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.320942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fb93a-093b-4cec-87ee-0b62e4940932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.321328 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb93a-093b-4cec-87ee-0b62e4940932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:23.321900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fb93a-093b-4cec-87ee-0b62e4940932 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:23.332614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1180/2362] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:45:23.336650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b9925-0c4e-4329-af6c-867abdb8a88d admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:23.347227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b9925-0c4e-4329-af6c-867abdb8a88d admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:23.351843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:23.352519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.352803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.353080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] PATH_INFO: `/v3/projects/5fd0a27a2f13490eb9d5a5b74956df31/users/f87cc3e4f0954c0388d277913bfe3285/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.369521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd0a27a2f13490eb9d5a5b74956df31, user_id=f87cc3e4f0954c0388d277913bfe3285, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:23.372278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:23.385518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1183/2363] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:45:23 2022] PUT /identity/v3/projects/5fd0a27a2f13490eb9d5a5b74956df31/users/f87cc3e4f0954c0388d277913bfe3285/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:45:23.390491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1c6d-b2eb-44c1-8c27-6e769cf84707 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.390794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1c6d-b2eb-44c1-8c27-6e769cf84707 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.391681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c1c6d-b2eb-44c1-8c27-6e769cf84707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.410705 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cb1c1c6d-b2eb-44c1-8c27-6e769cf84707 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:45:23.411908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cb1c1c6d-b2eb-44c1-8c27-6e769cf84707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:23.469945 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1181/2364] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:45:23.479482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:23.481277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:23.539691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81d613-0131-4b07-aabb-ebcc31622dd2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:23.543429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17534c2-0c79-4e39-8635-1d0aaa430d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:23.544037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17534c2-0c79-4e39-8635-1d0aaa430d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:23.544351 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17534c2-0c79-4e39-8635-1d0aaa430d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:23.544622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17534c2-0c79-4e39-8635-1d0aaa430d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:23.554372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17534c2-0c79-4e39-8635-1d0aaa430d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:23.555796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17534c2-0c79-4e39-8635-1d0aaa430d9d service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:23.565972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1184/2365] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:45:24.181174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a6f6-78de-419f-a56d-ff9c1e11febc None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:24.181686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a6f6-78de-419f-a56d-ff9c1e11febc None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:24.181786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6a6f6-78de-419f-a56d-ff9c1e11febc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:24.209765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-97c6a6f6-78de-419f-a56d-ff9c1e11febc None None] MFA Rules not processed for user `f87cc3e4f0954c0388d277913bfe3285`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:45:24.211062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-97c6a6f6-78de-419f-a56d-ff9c1e11febc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:24.230720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:45:24.279709 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1182/2366] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:45:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:45:24.284850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17534c2-0c79-4e39-8635-1d0aaa430d9d service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:24.294502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17534c2-0c79-4e39-8635-1d0aaa430d9d service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:24.299063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e2a90-7c3b-415a-89fd-1636b14cb8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:24.299643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e2a90-7c3b-415a-89fd-1636b14cb8d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:24.299918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e2a90-7c3b-415a-89fd-1636b14cb8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:24.300215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e2a90-7c3b-415a-89fd-1636b14cb8d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:24.300834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e2a90-7c3b-415a-89fd-1636b14cb8d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:24.302765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e2a90-7c3b-415a-89fd-1636b14cb8d1 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:24.335619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1185/2367] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:45:24 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:45:24.337097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6a6f6-78de-419f-a56d-ff9c1e11febc None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:24.350023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6a6f6-78de-419f-a56d-ff9c1e11febc None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:24.414988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f435e-3ebf-41bb-a6dd-53352bc50110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:24.415604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f435e-3ebf-41bb-a6dd-53352bc50110 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:24.415877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f435e-3ebf-41bb-a6dd-53352bc50110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:24.416238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f435e-3ebf-41bb-a6dd-53352bc50110 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:24.416867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f435e-3ebf-41bb-a6dd-53352bc50110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:24.419080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f435e-3ebf-41bb-a6dd-53352bc50110 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:24.441497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1183/2368] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:45:24 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:45:24.445521 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e2a90-7c3b-415a-89fd-1636b14cb8d1 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:24.455894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e2a90-7c3b-415a-89fd-1636b14cb8d1 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:24.460832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af36f2f-175c-4bcc-a921-18566a03f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:24.461370 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af36f2f-175c-4bcc-a921-18566a03f2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:24.461720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af36f2f-175c-4bcc-a921-18566a03f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:24.462020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af36f2f-175c-4bcc-a921-18566a03f2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:24.462719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af36f2f-175c-4bcc-a921-18566a03f2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:24.464304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af36f2f-175c-4bcc-a921-18566a03f2c2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:24.475990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1186/2369] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:45:24.480170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f435e-3ebf-41bb-a6dd-53352bc50110 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:24.491010 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f435e-3ebf-41bb-a6dd-53352bc50110 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:24.495673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ac30-d14e-482d-9a70-306c8440af2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:24.496361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ac30-d14e-482d-9a70-306c8440af2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:24.496643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ac30-d14e-482d-9a70-306c8440af2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:24.496918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ac30-d14e-482d-9a70-306c8440af2b admin admin] PATH_INFO: `/v3/projects/06f5a1e96f2b4ffcbf518de535440e33/users/d250554ebee44a679240cdf4eaf6542e/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:24.512130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916ac30-d14e-482d-9a70-306c8440af2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f5a1e96f2b4ffcbf518de535440e33, user_id=d250554ebee44a679240cdf4eaf6542e, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:24.514844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916ac30-d14e-482d-9a70-306c8440af2b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:24.528510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1184/2370] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:45:24 2022] PUT /identity/v3/projects/06f5a1e96f2b4ffcbf518de535440e33/users/d250554ebee44a679240cdf4eaf6542e/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:45:24.533099 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af36f2f-175c-4bcc-a921-18566a03f2c2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:24.542752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af36f2f-175c-4bcc-a921-18566a03f2c2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:24.575887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:45:24.602241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789ad6f-9e72-4c86-bd7a-ed9af7f9967d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:24.602830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ad6f-9e72-4c86-bd7a-ed9af7f9967d admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:24.603105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ad6f-9e72-4c86-bd7a-ed9af7f9967d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:24.603371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ad6f-9e72-4c86-bd7a-ed9af7f9967d admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:24.604002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789ad6f-9e72-4c86-bd7a-ed9af7f9967d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:24.605600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789ad6f-9e72-4c86-bd7a-ed9af7f9967d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:24.614998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1187/2371] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:45:24.619103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ac30-d14e-482d-9a70-306c8440af2b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:45:24.629255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ac30-d14e-482d-9a70-306c8440af2b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:45:24.633773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:45:24.634407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:24.634679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:24.634971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] PATH_INFO: `/v3/projects/06f5a1e96f2b4ffcbf518de535440e33/users/d250554ebee44a679240cdf4eaf6542e/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:24.636877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f5a1e96f2b4ffcbf518de535440e33, user_id=d250554ebee44a679240cdf4eaf6542e, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:45:24.639460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:45:24.650437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1185/2372] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:45:24 2022] PUT /identity/v3/projects/06f5a1e96f2b4ffcbf518de535440e33/users/d250554ebee44a679240cdf4eaf6542e/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:45:25.286244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6517-b82e-4e2b-94f4-e38eb9ccd402 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:45:25.286786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6517-b82e-4e2b-94f4-e38eb9ccd402 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:45:25.286905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6517-b82e-4e2b-94f4-e38eb9ccd402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:45:25.314875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1daf6517-b82e-4e2b-94f4-e38eb9ccd402 None None] MFA Rules not processed for user `d250554ebee44a679240cdf4eaf6542e`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:45:25.316125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1daf6517-b82e-4e2b-94f4-e38eb9ccd402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:45:25.386555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1188/2373] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:45:34.646634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:45:35.383557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=47 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:47:21.356252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:47:21.357898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:47:21.415665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5362e6-e110-41c4-8244-f9a55ab36dc1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:47:21.419279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:47:21.419877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:47:21.420188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:47:21.420460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:47:21.422154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:47:21.505369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:47:21.506794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:47:21.515688 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1186/2374] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:47:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:47:21.565037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf6517-b82e-4e2b-94f4-e38eb9ccd402 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:47:21.574438 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf6517-b82e-4e2b-94f4-e38eb9ccd402 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:47:21.577977 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:47:21.578572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:47:21.578850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:47:21.579121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:47:21.581033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:47:21.650189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:47:21.651623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:47:21.660698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1189/2375] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:47:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:47:21.708238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:47:21.716563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0889860-5fe1-4897-915f-413f4e544576 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:47:21.720174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:47:21.720964 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:47:21.721245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:47:21.721515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:47:21.777247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:47:21.778660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:47:21.787925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1187/2376] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:47:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:47:22.749436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:47:22.750834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:47:22.836112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719d14d-1ed3-43bb-809c-e99475733c93 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:47:22.841166 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:47:22.841764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:47:22.842044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:47:22.842314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:47:22.844009 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:47:22.856669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:47:22.858246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:47:22.868681 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1190/2377] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:47:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:47:31.784025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:47:32.864587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=72 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:48:18.573230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:48:18.574712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:48:18.682681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110ddc1-a163-4b6f-bc1c-3c05daf324fc service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:48:18.688580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:48:18.689706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:48:18.689988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:48:18.690255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:48:18.691967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:48:18.756743 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:48:18.758304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:48:18.776935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1188/2378] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:48:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:48:18.829109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:48:18.831101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:48:18.878162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:48:18.879544 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:48:18.894616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843c73b-6d2b-443e-a542-6f4a57b23d77 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:48:18.900148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:48:18.900751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:48:18.901023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:48:18.901289 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:48:18.903002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:48:18.915729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede51fbe-f710-40a9-9b8a-8337d766dc31 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:48:18.919353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:48:18.919950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:48:18.920257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:48:18.920531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:48:18.922262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:48:18.935474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:48:18.936919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:48:18.947882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:48:18.948632 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1191/2379] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:48:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:48:18.974997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:48:18.976622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:48:18.995439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1189/2380] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:48:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 16:48:28.943873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=39 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:48:28.992621 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=16 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:49:28.500966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:49:28.510021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5862fcb-3b31-4f9f-b69a-569a2edcc78b service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:49:28.513618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:49:28.514206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:49:28.514483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:49:28.514749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:49:28.516511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:49:28.526897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:49:28.528347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:49:28.536164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1192/2381] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:49:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:49:38.532784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=6 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:50:21.686991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:50:21.696487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0eecf-081e-4db4-9abb-f98b6a218562 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:50:21.699973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941f7ea-ab46-484d-aa44-04d9b8f56098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:50:21.700589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941f7ea-ab46-484d-aa44-04d9b8f56098 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:50:21.700866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941f7ea-ab46-484d-aa44-04d9b8f56098 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:50:21.701151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941f7ea-ab46-484d-aa44-04d9b8f56098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:50:21.702832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c941f7ea-ab46-484d-aa44-04d9b8f56098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:50:21.715570 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941f7ea-ab46-484d-aa44-04d9b8f56098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:50:21.716985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941f7ea-ab46-484d-aa44-04d9b8f56098 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:50:21.728619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1190/2382] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:50:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:50:31.722926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:51:19.789396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:19.790847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:19.811607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81fb63-8178-4898-a231-8918b45fd934 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:19.815660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:19.816291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:19.816565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:19.816848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:19.819098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:19.884241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:19.885814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:19.895364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1193/2383] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:51:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:51:21.531889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17ac4a-3b95-45b2-afc1-0cb8b1426ad8 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:21.532455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17ac4a-3b95-45b2-afc1-0cb8b1426ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:21.532455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17ac4a-3b95-45b2-afc1-0cb8b1426ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:21.560220 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-be17ac4a-3b95-45b2-afc1-0cb8b1426ad8 None None] MFA Rules not processed for user `60ee85a52c0244879f10e8943d20dfb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:51:21.561415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-be17ac4a-3b95-45b2-afc1-0cb8b1426ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:21.569074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1191/2384] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:51:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:51:21.871033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:21.880489 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fa091-baaf-4b72-9deb-6e6a5b6e1e17 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:21.884162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:21.884752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:21.885028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:21.885295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:21.886998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:21.896642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:21.898190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:21.905927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1194/2385] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:51:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:51:23.133656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17ac4a-3b95-45b2-afc1-0cb8b1426ad8 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:23.143506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17ac4a-3b95-45b2-afc1-0cb8b1426ad8 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:23.148173 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:23.148782 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:23.149055 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:23.149323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:23.151050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:23.216259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:23.217790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:23.228702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1192/2386] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:51:23.252633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:23.264873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d9bba-9090-48f5-9d21-6214269d3492 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:23.270027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:23.270615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:23.271192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:23.271460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:23.273208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:23.286016 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:23.288187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:23.298320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1195/2387] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:51:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:51:26.361108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:26.374708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471902b-efcd-4cf2-b44b-625f41b8d707 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:26.380615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:26.381699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:26.381981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:26.382255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:26.384891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:26.398964 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:26.400698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:26.414801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1193/2388] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:51:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:51:27.392590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:27.404729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2142-c769-4b16-bd48-ca646c0a7cf9 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:27.410150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:27.410774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:27.411058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:27.411328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:27.413387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:27.426115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:27.428984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:27.441908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1196/2389] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:51:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:51:30.938753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:30.941154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:30.957944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703a332-fdd7-43a8-9ccb-dd133cab4204 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:30.972472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:30.973097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:30.973503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:30.973778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:30.975968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:30.988662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:30.991456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:31.003236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1194/2390] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:51:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:51:31.270266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:31.272418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:31.286992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a98d4-9cd6-43ae-9029-976855f51fed service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:31.291972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:31.292703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:31.292918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:31.293187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] PATH_INFO: `/v3/users/ea2b7595ca6d446c84e95a05a701e9dc` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:31.294345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2b7595ca6d446c84e95a05a701e9dc)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:31.297017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:31.347493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2b7595ca6d446c84e95a05a701e9dc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:31.362484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2b7595ca6d446c84e95a05a701e9dc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:31.363229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2b7595ca6d446c84e95a05a701e9dc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:31.372893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2b7595ca6d446c84e95a05a701e9dc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:31.383742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2b7595ca6d446c84e95a05a701e9dc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:31.394539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1197/2391] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:51:31 2022] DELETE /identity/v3/users/ea2b7595ca6d446c84e95a05a701e9dc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:51:31.628830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:31.642265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878e961-54ea-4d10-9f9a-e8edc28b4eac service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:31.679576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:31.682699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:31.720726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:51:31.779751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:31.780447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:31.780729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:31.780999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] PATH_INFO: `/v3/projects/ab0c02c8d1fa4a9fbbb64a655643ff31` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:31.796740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92eac59-fd70-49c2-8a33-72d29956757e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:31.807192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:31.808831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:31.809392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:31.809878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:31.812253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:31.816119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0c02c8d1fa4a9fbbb64a655643ff31)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:31.818671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:31.833660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:31.835660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:31.857677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1198/2392] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:51:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:51:31.926465 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:51:31.929918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0c02c8d1fa4a9fbbb64a655643ff31', 'request_id': 'req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:31.942567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0c02c8d1fa4a9fbbb64a655643ff31', 'request_id': 'req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:31.985105 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0c02c8d1fa4a9fbbb64a655643ff31', 'request_id': 'req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.002875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] The token cache is being invalidate because project ab0c02c8d1fa4a9fbbb64a655643ff31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:51:32.003163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.004158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.006839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1195/2393] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:51:31 2022] DELETE /identity/v3/projects/ab0c02c8d1fa4a9fbbb64a655643ff31 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:51:32.011567 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:32.014161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:32.030666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e413b-e707-4e51-9841-282c62138797 service nova] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:32.110027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:32.110027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:32.110027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:32.110553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] PATH_INFO: `/v3/users/ba8034ff9f7f4af89a68c59e72ce02d4` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:32.131299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8034ff9f7f4af89a68c59e72ce02d4)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:32.133348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:32.202285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8034ff9f7f4af89a68c59e72ce02d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.213770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8034ff9f7f4af89a68c59e72ce02d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.214457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8034ff9f7f4af89a68c59e72ce02d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.221059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8034ff9f7f4af89a68c59e72ce02d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.228948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8034ff9f7f4af89a68c59e72ce02d4'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.235653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1199/2394] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:51:32 2022] DELETE /identity/v3/users/ba8034ff9f7f4af89a68c59e72ce02d4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:51:32.408808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:51:32.410270 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:51:32.424564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050bd10-2b2d-42a8-8e40-c5f59ead2d59 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:51:32.494444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:51:32.495068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:51:32.495342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:51:32.495609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] PATH_INFO: `/v3/projects/143280ba084940e0b9bb18410615a201` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:51:32.506459 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=143280ba084940e0b9bb18410615a201)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:51:32.508389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:51:32.553120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '143280ba084940e0b9bb18410615a201', 'request_id': 'req-107833bd-2331-4026-95a8-316652ce75e0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.560115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '143280ba084940e0b9bb18410615a201', 'request_id': 'req-107833bd-2331-4026-95a8-316652ce75e0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.568741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '143280ba084940e0b9bb18410615a201', 'request_id': 'req-107833bd-2331-4026-95a8-316652ce75e0'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.578803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] The token cache is being invalidate because project 143280ba084940e0b9bb18410615a201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:51:32.579089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.579714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:51:32.581241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1196/2395] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:51:32 2022] DELETE /identity/v3/projects/143280ba084940e0b9bb18410615a201 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:51:42.231486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:51:42.575835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=62,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:52:21.455942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:52:21.458383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:52:21.545287 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f84695-4b3c-49fd-927c-a8b0ece247b7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:52:21.607270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:52:21.607875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:52:21.608188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:52:21.608462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:52:21.610190 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:52:21.675760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:52:21.677502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:52:21.696001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1200/2396] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:52:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:52:28.830369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:52:28.831804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:52:28.918626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107833bd-2331-4026-95a8-316652ce75e0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:52:28.923560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:52:28.924187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:52:28.924463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:52:28.924733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:52:28.926994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:52:28.937911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:52:28.939465 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:52:28.949847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1197/2397] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:52:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:52:31.693548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:52:38.945807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=37 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:53:18.185789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:53:18.187279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:53:18.245020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df003b-c32f-4b47-8803-8d4dd2ea743e service nova] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:53:18.248621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:53:18.249222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:53:18.249500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:53:18.249770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:53:18.251503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:53:18.278010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:53:18.279441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:53:18.290324 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1201/2398] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:53:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:53:19.949564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:53:19.950961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:53:20.018771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eed23d-1e18-43ed-8df9-bfe5909974e7 service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:53:20.022566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:53:20.023163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:53:20.023441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:53:20.023713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:53:20.025452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:53:20.095229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:53:20.096630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:53:20.106206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1198/2399] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:53:19 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:53:20.168738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:53:20.178850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fdae1-87d5-4035-b22e-8c0028e9144f service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:53:20.182451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:53:20.183037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:53:20.183312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:53:20.183577 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:53:20.185299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:53:20.243570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:53:20.245047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:53:20.255063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1202/2400] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:53:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:53:20.296903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:53:20.305955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef262b0f-a969-4a0c-8a57-5eac11dabfe8 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:53:20.309643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:53:20.310242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:53:20.310518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:53:20.310788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:53:20.312543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:53:20.385130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:53:20.387789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:53:20.398442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1199/2401] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:53:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:53:30.250605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=59 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:53:30.393075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:54:31.094657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:54:31.096205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:54:31.120295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4874fb1-1033-4706-9cb2-4487b03c7e7d service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:54:31.124198 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:54:31.124798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:54:31.125074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:54:31.125343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:54:31.127608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:54:31.136542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:54:31.138491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:54:31.147807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1203/2402] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:54:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:54:41.143959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=11 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:55:20.537336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:20.546203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c584d-13bc-4b33-8f6e-57672979a27f service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:20.550470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:20.551077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:20.551356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:20.551783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:20.553574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] Could not find project: ab0c02c8d1fa4a9fbbb64a655643ff31.: keystone.exception.ProjectNotFound: Could not find project: ab0c02c8d1fa4a9fbbb64a655643ff31. Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 11 16:55:20.566575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:55:20.568081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 11 16:55:20.569250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.get_or_create( Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application with Lock( Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._enter() Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.creator() Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application created_value = creator( Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 11 16:55:20.570374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 11 16:55:20.571433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: ab0c02c8d1fa4a9fbbb64a655643ff31. Aug 11 16:55:20.571433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: ERROR keystone.server.flask.application Aug 11 16:55:20.571433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1200/2403] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:55:20 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 11 16:55:21.670025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:21.678622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2899a-d444-45e2-89f9-7b8a4690f4f8 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:21.682223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793151c-ce46-4618-b8d9-e0946e7b5d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:21.682807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793151c-ce46-4618-b8d9-e0946e7b5d4d service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:21.683089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793151c-ce46-4618-b8d9-e0946e7b5d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:21.683356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793151c-ce46-4618-b8d9-e0946e7b5d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:21.685078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f793151c-ce46-4618-b8d9-e0946e7b5d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:21.698268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793151c-ce46-4618-b8d9-e0946e7b5d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:21.699718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793151c-ce46-4618-b8d9-e0946e7b5d4d service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:21.710836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1204/2404] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:55:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:55:23.760837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:23.762491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:23.869340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dd000-217d-41d3-a03f-7bdc04d8d681 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:23.874764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:23.875393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:23.876455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:23.876455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] PATH_INFO: `/v3/users/86bee9d5cd1047e28ed3f61385c886f5` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:23.894273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86bee9d5cd1047e28ed3f61385c886f5)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:23.896486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:23.941755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86bee9d5cd1047e28ed3f61385c886f5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:23.949532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86bee9d5cd1047e28ed3f61385c886f5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:23.950165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86bee9d5cd1047e28ed3f61385c886f5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:23.962452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86bee9d5cd1047e28ed3f61385c886f5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:23.963072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86bee9d5cd1047e28ed3f61385c886f5'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:23.975343 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1201/2405] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:55:23 2022] DELETE /identity/v3/users/86bee9d5cd1047e28ed3f61385c886f5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:55:23.979780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690bb11-4b0f-4fc2-aae7-37ed4d9be491 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:23.980296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690bb11-4b0f-4fc2-aae7-37ed4d9be491 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:23.980405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690bb11-4b0f-4fc2-aae7-37ed4d9be491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:24.007739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-b690bb11-4b0f-4fc2-aae7-37ed4d9be491 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:55:24.008966 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b690bb11-4b0f-4fc2-aae7-37ed4d9be491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:24.067749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1205/2406] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:55:24.077392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:24.079496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:24.114498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:55:24.139349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4cb53-9842-4e5e-b473-43e6a3c37975 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:24.143741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fcfaa-6836-4b75-af21-c4bcd9951923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:24.144383 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fcfaa-6836-4b75-af21-c4bcd9951923 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:24.144661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fcfaa-6836-4b75-af21-c4bcd9951923 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:24.144932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fcfaa-6836-4b75-af21-c4bcd9951923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:24.153660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fcfaa-6836-4b75-af21-c4bcd9951923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:24.155097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fcfaa-6836-4b75-af21-c4bcd9951923 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:24.165077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1202/2407] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:55:24.440226 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690bb11-4b0f-4fc2-aae7-37ed4d9be491 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:24.441798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-b690bb11-4b0f-4fc2-aae7-37ed4d9be491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:24.455313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690bb11-4b0f-4fc2-aae7-37ed4d9be491 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:24.460589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:24.461246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:24.461546 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:24.461804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] PATH_INFO: `/v3/projects/07e61311e1d347a5960deb9582bf43c2` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:24.473845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] RBAC: Authorizing `identity:delete_project(project_id=07e61311e1d347a5960deb9582bf43c2)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:24.475839 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:24.522768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07e61311e1d347a5960deb9582bf43c2', 'request_id': 'req-5c4f4158-39d2-48eb-aef3-2a4720b9307b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.530495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07e61311e1d347a5960deb9582bf43c2', 'request_id': 'req-5c4f4158-39d2-48eb-aef3-2a4720b9307b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.536585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07e61311e1d347a5960deb9582bf43c2', 'request_id': 'req-5c4f4158-39d2-48eb-aef3-2a4720b9307b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.549037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] The token cache is being invalidate because project 07e61311e1d347a5960deb9582bf43c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:55:24.549325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.549951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.551422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1206/2408] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:55:24 2022] DELETE /identity/v3/projects/07e61311e1d347a5960deb9582bf43c2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:55:24.555567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fcfaa-6836-4b75-af21-c4bcd9951923 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:24.565366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fcfaa-6836-4b75-af21-c4bcd9951923 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:24.621693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:24.622318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:24.622595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:24.622883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] PATH_INFO: `/v3/users/5bd7fe7cdbfe492cb38f9e640e090c14` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:24.640539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd7fe7cdbfe492cb38f9e640e090c14)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:24.643265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:24.687894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd7fe7cdbfe492cb38f9e640e090c14'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.694785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd7fe7cdbfe492cb38f9e640e090c14'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.695375 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd7fe7cdbfe492cb38f9e640e090c14'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.707998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd7fe7cdbfe492cb38f9e640e090c14'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.708656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-b4dca608-5ea7-4219-914f-c6b1464d6fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd7fe7cdbfe492cb38f9e640e090c14'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:24.721054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1203/2409] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:55:24 2022] DELETE /identity/v3/users/5bd7fe7cdbfe492cb38f9e640e090c14 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:55:24.989197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:24.990705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:25.010185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f4158-39d2-48eb-aef3-2a4720b9307b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.060518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:55:25.074767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.075822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.076501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.076811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] PATH_INFO: `/v3/projects/974ecdcb9c314071bca1d9629b73ac36` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.089294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] RBAC: Authorizing `identity:delete_project(project_id=974ecdcb9c314071bca1d9629b73ac36)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.091282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.136026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974ecdcb9c314071bca1d9629b73ac36', 'request_id': 'req-48ca78e8-7e2b-429f-adb2-697d7935882c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:25.144243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974ecdcb9c314071bca1d9629b73ac36', 'request_id': 'req-48ca78e8-7e2b-429f-adb2-697d7935882c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:25.149921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974ecdcb9c314071bca1d9629b73ac36', 'request_id': 'req-48ca78e8-7e2b-429f-adb2-697d7935882c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:25.161869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] The token cache is being invalidate because project 974ecdcb9c314071bca1d9629b73ac36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:55:25.162152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:25.162761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-48ca78e8-7e2b-429f-adb2-697d7935882c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:55:25.164243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1207/2410] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:55:24 2022] DELETE /identity/v3/projects/974ecdcb9c314071bca1d9629b73ac36 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:55:25.169065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b0377-4938-4a0d-9ee1-d8348a30204b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.169538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b0377-4938-4a0d-9ee1-d8348a30204b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.169637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b0377-4938-4a0d-9ee1-d8348a30204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.217341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-4b5b0377-4938-4a0d-9ee1-d8348a30204b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:55:25.218528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-4b5b0377-4938-4a0d-9ee1-d8348a30204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:25.229995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:55:25.277615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1204/2411] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:55:25.285548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc625dc-6d7a-4586-acf3-5c8acf94275b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.285833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc625dc-6d7a-4586-acf3-5c8acf94275b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.286112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc625dc-6d7a-4586-acf3-5c8acf94275b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.305519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3dc625dc-6d7a-4586-acf3-5c8acf94275b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:55:25.306722 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3dc625dc-6d7a-4586-acf3-5c8acf94275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:25.315273 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1208/2412] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:55:25.319631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b0377-4938-4a0d-9ee1-d8348a30204b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.329316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b0377-4938-4a0d-9ee1-d8348a30204b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.333786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.334374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.334648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.334915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.335545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:55:25.335838 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.337447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.348702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1205/2413] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:55:25.352788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc625dc-6d7a-4586-acf3-5c8acf94275b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.362428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc625dc-6d7a-4586-acf3-5c8acf94275b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.367040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176468-37da-4238-85af-81b504eedff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.367628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176468-37da-4238-85af-81b504eedff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.367905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176468-37da-4238-85af-81b504eedff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.368206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176468-37da-4238-85af-81b504eedff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.368825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5176468-37da-4238-85af-81b504eedff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.370702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5176468-37da-4238-85af-81b504eedff2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.407662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1209/2414] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:55:25 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:55:25.411482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.421377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ac53-866a-4ee1-8eef-bc03e64cea68 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.476956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e5eb-de6d-4569-b19b-1c07ddf8e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.477549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e5eb-de6d-4569-b19b-1c07ddf8e9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.477825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e5eb-de6d-4569-b19b-1c07ddf8e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.478093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48e5eb-de6d-4569-b19b-1c07ddf8e9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.478718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e5eb-de6d-4569-b19b-1c07ddf8e9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.480654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48e5eb-de6d-4569-b19b-1c07ddf8e9a2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.501215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1206/2415] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:55:25 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:55:25.505364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176468-37da-4238-85af-81b504eedff2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.515765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176468-37da-4238-85af-81b504eedff2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.521102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d1ca0-f731-4e02-9534-f7f322faec6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.521694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d1ca0-f731-4e02-9534-f7f322faec6e admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.521972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d1ca0-f731-4e02-9534-f7f322faec6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.522244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d1ca0-f731-4e02-9534-f7f322faec6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.522885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d1ca0-f731-4e02-9534-f7f322faec6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.524537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d1ca0-f731-4e02-9534-f7f322faec6e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.534871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1210/2416] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:55:25.538885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e5eb-de6d-4569-b19b-1c07ddf8e9a2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.549114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48e5eb-de6d-4569-b19b-1c07ddf8e9a2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.553786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176132e3-d975-4908-9396-6d2485642978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.554381 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176132e3-d975-4908-9396-6d2485642978 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.554659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176132e3-d975-4908-9396-6d2485642978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.554929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176132e3-d975-4908-9396-6d2485642978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.555596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176132e3-d975-4908-9396-6d2485642978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.557213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176132e3-d975-4908-9396-6d2485642978 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.568028 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1207/2417] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:55:25.572052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d1ca0-f731-4e02-9534-f7f322faec6e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.582612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d1ca0-f731-4e02-9534-f7f322faec6e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.587101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae5509-de29-4344-b238-87b23105e91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.587728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae5509-de29-4344-b238-87b23105e91a admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.588003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae5509-de29-4344-b238-87b23105e91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.588300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae5509-de29-4344-b238-87b23105e91a admin admin] PATH_INFO: `/v3/projects/71dd6a1547244a0a9eecfdde2126e8fa/users/7e09486ae0f24e66b92775153721836f/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.603145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae5509-de29-4344-b238-87b23105e91a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71dd6a1547244a0a9eecfdde2126e8fa, user_id=7e09486ae0f24e66b92775153721836f, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.605795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae5509-de29-4344-b238-87b23105e91a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.618678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1211/2418] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:55:25 2022] PUT /identity/v3/projects/71dd6a1547244a0a9eecfdde2126e8fa/users/7e09486ae0f24e66b92775153721836f/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:55:25.630369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071368c-965c-4ed5-8295-6c6eef8aabf5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.630675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071368c-965c-4ed5-8295-6c6eef8aabf5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.630941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071368c-965c-4ed5-8295-6c6eef8aabf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.659481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-3071368c-965c-4ed5-8295-6c6eef8aabf5 None None] MFA Rules not processed for user `7e09486ae0f24e66b92775153721836f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:55:25.660730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3071368c-965c-4ed5-8295-6c6eef8aabf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:25.723813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1208/2419] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:55:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:55:25.728347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae5509-de29-4344-b238-87b23105e91a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.789668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae5509-de29-4344-b238-87b23105e91a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.794486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deacb68-ca6d-4e36-8a91-f7af8f7d57af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.795088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deacb68-ca6d-4e36-8a91-f7af8f7d57af admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.795366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deacb68-ca6d-4e36-8a91-f7af8f7d57af admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.795636 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deacb68-ca6d-4e36-8a91-f7af8f7d57af admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.796326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deacb68-ca6d-4e36-8a91-f7af8f7d57af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.798920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deacb68-ca6d-4e36-8a91-f7af8f7d57af admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.808180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:55:25.829294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1212/2420] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:55:25 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:55:25.833486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071368c-965c-4ed5-8295-6c6eef8aabf5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.844713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071368c-965c-4ed5-8295-6c6eef8aabf5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.853629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:55:25.900898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4a4d8-d51a-47bb-882d-e945569f0f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.901501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a4d8-d51a-47bb-882d-e945569f0f8a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.901775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a4d8-d51a-47bb-882d-e945569f0f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.902044 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a4d8-d51a-47bb-882d-e945569f0f8a admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.902664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4a4d8-d51a-47bb-882d-e945569f0f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.904604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4a4d8-d51a-47bb-882d-e945569f0f8a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.925899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1209/2421] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:55:25 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:55:25.929892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deacb68-ca6d-4e36-8a91-f7af8f7d57af admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.940755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deacb68-ca6d-4e36-8a91-f7af8f7d57af admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.945595 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bf7d3-050a-48e2-9a55-9682339effed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.946183 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bf7d3-050a-48e2-9a55-9682339effed admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.946462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bf7d3-050a-48e2-9a55-9682339effed admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.946730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bf7d3-050a-48e2-9a55-9682339effed admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.947366 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bf7d3-050a-48e2-9a55-9682339effed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.949011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bf7d3-050a-48e2-9a55-9682339effed admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:25.958942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1213/2422] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:55:25.963020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4a4d8-d51a-47bb-882d-e945569f0f8a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:25.973752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4a4d8-d51a-47bb-882d-e945569f0f8a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:25.978384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96689e77-bd45-420f-88a8-1ff6ac1005ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:25.979055 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96689e77-bd45-420f-88a8-1ff6ac1005ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:25.979331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96689e77-bd45-420f-88a8-1ff6ac1005ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:25.979604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96689e77-bd45-420f-88a8-1ff6ac1005ad admin admin] PATH_INFO: `/v3/projects/c12940af0a6b4e8cae7ad5e469223d1e/users/58c5ee4bceec4eb59653c927f0954746/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:25.994728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96689e77-bd45-420f-88a8-1ff6ac1005ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12940af0a6b4e8cae7ad5e469223d1e, user_id=58c5ee4bceec4eb59653c927f0954746, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:25.997398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96689e77-bd45-420f-88a8-1ff6ac1005ad admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:26.010237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1210/2423] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:55:25 2022] PUT /identity/v3/projects/c12940af0a6b4e8cae7ad5e469223d1e/users/58c5ee4bceec4eb59653c927f0954746/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:55:26.014176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bf7d3-050a-48e2-9a55-9682339effed admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:26.024666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bf7d3-050a-48e2-9a55-9682339effed admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:26.082804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b59ca1-8fab-4aa2-bf61-80960badf4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:26.083434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b59ca1-8fab-4aa2-bf61-80960badf4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:26.083768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b59ca1-8fab-4aa2-bf61-80960badf4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:26.084017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b59ca1-8fab-4aa2-bf61-80960badf4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:26.084716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b59ca1-8fab-4aa2-bf61-80960badf4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:26.086347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b59ca1-8fab-4aa2-bf61-80960badf4c2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:26.096463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1214/2424] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:55:26.100632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96689e77-bd45-420f-88a8-1ff6ac1005ad admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:55:26.110829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96689e77-bd45-420f-88a8-1ff6ac1005ad admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:55:26.115425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:55:26.116126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:26.116480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:26.116757 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] PATH_INFO: `/v3/projects/c12940af0a6b4e8cae7ad5e469223d1e/users/58c5ee4bceec4eb59653c927f0954746/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:26.118656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12940af0a6b4e8cae7ad5e469223d1e, user_id=58c5ee4bceec4eb59653c927f0954746, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:55:26.121324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:55:26.131257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1211/2425] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:55:26 2022] PUT /identity/v3/projects/c12940af0a6b4e8cae7ad5e469223d1e/users/58c5ee4bceec4eb59653c927f0954746/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:55:26.141766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf50f-7ed7-4dc5-9fa8-c4fda13516e5 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:55:26.142048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf50f-7ed7-4dc5-9fa8-c4fda13516e5 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:55:26.142318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf50f-7ed7-4dc5-9fa8-c4fda13516e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:55:26.169589 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-675cf50f-7ed7-4dc5-9fa8-c4fda13516e5 None None] MFA Rules not processed for user `58c5ee4bceec4eb59653c927f0954746`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:55:26.170783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-675cf50f-7ed7-4dc5-9fa8-c4fda13516e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:55:26.236050 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1215/2426] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:55:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:55:36.128380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=38,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:55:36.233140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:57:20.899816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:20.901677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:20.958811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4f32f-89dc-45a3-9190-a50110bc191c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:20.962425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:20.963019 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:20.963291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:20.963556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:20.965247 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:21.021823 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:21.023227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:21.031987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1212/2427] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:57:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:57:21.074945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cf50f-7ed7-4dc5-9fa8-c4fda13516e5 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:21.085207 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cf50f-7ed7-4dc5-9fa8-c4fda13516e5 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:21.089516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c01c94-53ff-4956-b3c6-c003b25fd739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:21.090121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c01c94-53ff-4956-b3c6-c003b25fd739 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:21.090403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c01c94-53ff-4956-b3c6-c003b25fd739 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:21.090677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c01c94-53ff-4956-b3c6-c003b25fd739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:21.100334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c01c94-53ff-4956-b3c6-c003b25fd739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:21.101765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c01c94-53ff-4956-b3c6-c003b25fd739 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:21.111057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1216/2428] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:57:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:57:21.315835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:21.378617 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef225e3-d09b-438c-95e8-5ce5f68cf9a6 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:21.383159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:21.383774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:21.384046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:21.384340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] PATH_INFO: `/v3/users/7e09486ae0f24e66b92775153721836f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:21.385438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e09486ae0f24e66b92775153721836f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:21.387366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:21.432577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e09486ae0f24e66b92775153721836f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:21.439418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e09486ae0f24e66b92775153721836f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:21.440027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e09486ae0f24e66b92775153721836f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:21.441946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:57:21.454594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e09486ae0f24e66b92775153721836f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:21.455760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e09486ae0f24e66b92775153721836f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:21.467754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1213/2429] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:57:21 2022] DELETE /identity/v3/users/7e09486ae0f24e66b92775153721836f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:57:21.472983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3bf6-4e6a-445a-abab-1f47bca30c6c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:21.473309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3bf6-4e6a-445a-abab-1f47bca30c6c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:21.473576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3bf6-4e6a-445a-abab-1f47bca30c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:21.493037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1a9f3bf6-4e6a-445a-abab-1f47bca30c6c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:57:21.494235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1a9f3bf6-4e6a-445a-abab-1f47bca30c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:21.567677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1217/2430] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:57:21.577744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:21.579154 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:21.636163 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acf9f5-c246-4953-a171-a4cfb00edee5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:21.639818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b2e85-0eda-4e9b-8191-0a7c4fe21210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:21.640447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2e85-0eda-4e9b-8191-0a7c4fe21210 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:21.640727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2e85-0eda-4e9b-8191-0a7c4fe21210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:21.640999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b2e85-0eda-4e9b-8191-0a7c4fe21210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:21.649876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b2e85-0eda-4e9b-8191-0a7c4fe21210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:21.651286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b2e85-0eda-4e9b-8191-0a7c4fe21210 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:21.661181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1214/2431] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:57:21.930783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3bf6-4e6a-445a-abab-1f47bca30c6c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:21.932252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1a9f3bf6-4e6a-445a-abab-1f47bca30c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:21.945891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3bf6-4e6a-445a-abab-1f47bca30c6c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:21.950956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:21.952547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:21.952843 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:21.953461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] PATH_INFO: `/v3/projects/71dd6a1547244a0a9eecfdde2126e8fa` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:21.954624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71dd6a1547244a0a9eecfdde2126e8fa)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:21.957945 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:22.006002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71dd6a1547244a0a9eecfdde2126e8fa', 'request_id': 'req-843b1ed7-0949-41ae-a9ee-a2c947064cc0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.012076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71dd6a1547244a0a9eecfdde2126e8fa', 'request_id': 'req-843b1ed7-0949-41ae-a9ee-a2c947064cc0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.017557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71dd6a1547244a0a9eecfdde2126e8fa', 'request_id': 'req-843b1ed7-0949-41ae-a9ee-a2c947064cc0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.029996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] The token cache is being invalidate because project 71dd6a1547244a0a9eecfdde2126e8fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:57:22.030283 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.030926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.032411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1218/2432] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:57:21 2022] DELETE /identity/v3/projects/71dd6a1547244a0a9eecfdde2126e8fa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:57:22.036860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b2e85-0eda-4e9b-8191-0a7c4fe21210 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:22.046764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b2e85-0eda-4e9b-8191-0a7c4fe21210 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:22.103324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:22.103876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.104183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.104456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] PATH_INFO: `/v3/users/58c5ee4bceec4eb59653c927f0954746` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.105561 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c5ee4bceec4eb59653c927f0954746)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:22.107447 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:22.151187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c5ee4bceec4eb59653c927f0954746'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.159909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c5ee4bceec4eb59653c927f0954746'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.160682 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c5ee4bceec4eb59653c927f0954746'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.174445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c5ee4bceec4eb59653c927f0954746'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.175174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-4972dba9-4a13-4c0f-a35c-4b28c7655913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c5ee4bceec4eb59653c927f0954746'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.180760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:57:22.188999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1215/2433] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:57:22 2022] DELETE /identity/v3/users/58c5ee4bceec4eb59653c927f0954746 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:57:22.463436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:22.464929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:22.480660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b1ed7-0949-41ae-a9ee-a2c947064cc0 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:22.531507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:57:22.538581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:22.539623 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.539783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.540126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] PATH_INFO: `/v3/projects/c12940af0a6b4e8cae7ad5e469223d1e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.541371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12940af0a6b4e8cae7ad5e469223d1e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:22.543469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:22.588387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12940af0a6b4e8cae7ad5e469223d1e', 'request_id': 'req-3644123b-7372-475c-ac60-68b2ee6e5779'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.594289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12940af0a6b4e8cae7ad5e469223d1e', 'request_id': 'req-3644123b-7372-475c-ac60-68b2ee6e5779'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.600483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12940af0a6b4e8cae7ad5e469223d1e', 'request_id': 'req-3644123b-7372-475c-ac60-68b2ee6e5779'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.613919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] The token cache is being invalidate because project c12940af0a6b4e8cae7ad5e469223d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:57:22.614212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.614836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-3644123b-7372-475c-ac60-68b2ee6e5779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:57:22.616494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1219/2434] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:57:22 2022] DELETE /identity/v3/projects/c12940af0a6b4e8cae7ad5e469223d1e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:57:22.621306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1378-b8a9-4a44-9bb2-f4b4469cd85f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.621594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1378-b8a9-4a44-9bb2-f4b4469cd85f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.621870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1378-b8a9-4a44-9bb2-f4b4469cd85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.641261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-010a1378-b8a9-4a44-9bb2-f4b4469cd85f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:57:22.642929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-010a1378-b8a9-4a44-9bb2-f4b4469cd85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:22.701922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1216/2435] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:57:22.710382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201becb2-edee-428b-8d1a-4629eecc9fa4 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.710668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201becb2-edee-428b-8d1a-4629eecc9fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.710942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201becb2-edee-428b-8d1a-4629eecc9fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.731649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-201becb2-edee-428b-8d1a-4629eecc9fa4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:57:22.732905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-201becb2-edee-428b-8d1a-4629eecc9fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:22.741328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1220/2436] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:57:22.745559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a1378-b8a9-4a44-9bb2-f4b4469cd85f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:22.756361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a1378-b8a9-4a44-9bb2-f4b4469cd85f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:22.761070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:22.761663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.761962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.762230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.762869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:57:22.763167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:22.764812 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:22.776120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1217/2437] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 16:57:22.780229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201becb2-edee-428b-8d1a-4629eecc9fa4 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:22.789685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201becb2-edee-428b-8d1a-4629eecc9fa4 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:22.794375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac18c1-aadb-4580-96e3-6be006fde025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:22.794962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac18c1-aadb-4580-96e3-6be006fde025 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.795242 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac18c1-aadb-4580-96e3-6be006fde025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.795513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac18c1-aadb-4580-96e3-6be006fde025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.796158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac18c1-aadb-4580-96e3-6be006fde025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:22.798049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac18c1-aadb-4580-96e3-6be006fde025 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:22.825880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1221/2438] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:57:22 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:57:22.830691 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:22.847781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2cd2b-2b2f-45a2-a048-4defbb7c3d1d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:22.903146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d8b95-a205-4e11-bfba-bbac4a2c09b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:22.903668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d8b95-a205-4e11-bfba-bbac4a2c09b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.903941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d8b95-a205-4e11-bfba-bbac4a2c09b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.904234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d8b95-a205-4e11-bfba-bbac4a2c09b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.904838 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d8b95-a205-4e11-bfba-bbac4a2c09b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:22.906891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d8b95-a205-4e11-bfba-bbac4a2c09b0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:22.928018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1218/2439] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:57:22 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:57:22.932146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac18c1-aadb-4580-96e3-6be006fde025 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:22.950121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac18c1-aadb-4580-96e3-6be006fde025 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:22.954691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3a5a7-3de3-475e-beab-36bda1f519a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:22.955819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3a5a7-3de3-475e-beab-36bda1f519a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.956088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3a5a7-3de3-475e-beab-36bda1f519a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.956381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3a5a7-3de3-475e-beab-36bda1f519a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.957042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3a5a7-3de3-475e-beab-36bda1f519a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:22.958611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3a5a7-3de3-475e-beab-36bda1f519a6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:22.968517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1222/2440] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:57:22.972656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d8b95-a205-4e11-bfba-bbac4a2c09b0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:22.982546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d8b95-a205-4e11-bfba-bbac4a2c09b0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:22.987309 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357d20f-2aa2-4857-aed6-83de89cceb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:22.987904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357d20f-2aa2-4857-aed6-83de89cceb6f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:22.988211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357d20f-2aa2-4857-aed6-83de89cceb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:22.988489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357d20f-2aa2-4857-aed6-83de89cceb6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:22.989130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357d20f-2aa2-4857-aed6-83de89cceb6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:22.990713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357d20f-2aa2-4857-aed6-83de89cceb6f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:23.001296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1219/2441] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:57:23.005136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3a5a7-3de3-475e-beab-36bda1f519a6 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:23.014853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3a5a7-3de3-475e-beab-36bda1f519a6 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:23.020186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23dc64-a881-4a5a-b231-33789355d875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:23.020820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23dc64-a881-4a5a-b231-33789355d875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.021097 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23dc64-a881-4a5a-b231-33789355d875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.021369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23dc64-a881-4a5a-b231-33789355d875 admin admin] PATH_INFO: `/v3/projects/943727c0b4394cc78960ffd4cee20877/users/a811bd2dad9e4ec393a31541de13a912/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.035729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23dc64-a881-4a5a-b231-33789355d875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=943727c0b4394cc78960ffd4cee20877, user_id=a811bd2dad9e4ec393a31541de13a912, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:23.038397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23dc64-a881-4a5a-b231-33789355d875 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:23.051401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1223/2442] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:57:23 2022] PUT /identity/v3/projects/943727c0b4394cc78960ffd4cee20877/users/a811bd2dad9e4ec393a31541de13a912/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:57:23.062704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce873df-be64-4fc4-ade9-2fd4f6457b38 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.063174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce873df-be64-4fc4-ade9-2fd4f6457b38 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.063294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce873df-be64-4fc4-ade9-2fd4f6457b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.092901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7ce873df-be64-4fc4-ade9-2fd4f6457b38 None None] MFA Rules not processed for user `a811bd2dad9e4ec393a31541de13a912`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:57:23.094107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7ce873df-be64-4fc4-ade9-2fd4f6457b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:23.137243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:57:23.160190 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1220/2443] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:57:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:57:23.164700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23dc64-a881-4a5a-b231-33789355d875 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:23.224908 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23dc64-a881-4a5a-b231-33789355d875 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:23.229559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c3083-70b8-4115-b5e8-f6becf158203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:23.230146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c3083-70b8-4115-b5e8-f6becf158203 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.230425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c3083-70b8-4115-b5e8-f6becf158203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.230693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c3083-70b8-4115-b5e8-f6becf158203 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.231303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c3083-70b8-4115-b5e8-f6becf158203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:23.233233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c3083-70b8-4115-b5e8-f6becf158203 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:23.259170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1224/2444] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:57:23 2022] POST /identity/v3/projects => generated 389 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:57:23.263393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce873df-be64-4fc4-ade9-2fd4f6457b38 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:23.273911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce873df-be64-4fc4-ade9-2fd4f6457b38 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:23.329409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41761e19-646a-400a-a179-fd1c88f56676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:23.330037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41761e19-646a-400a-a179-fd1c88f56676 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.330311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41761e19-646a-400a-a179-fd1c88f56676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.330580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41761e19-646a-400a-a179-fd1c88f56676 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.331207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41761e19-646a-400a-a179-fd1c88f56676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:23.333140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41761e19-646a-400a-a179-fd1c88f56676 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:23.357268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1221/2445] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:57:23 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:57:23.361961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c3083-70b8-4115-b5e8-f6becf158203 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:23.367430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:57:23.373475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c3083-70b8-4115-b5e8-f6becf158203 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:23.378942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeba57-42e2-490b-9b39-40a396bc7ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:23.379534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeba57-42e2-490b-9b39-40a396bc7ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.379812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeba57-42e2-490b-9b39-40a396bc7ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.380084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeeba57-42e2-490b-9b39-40a396bc7ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.380757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeeba57-42e2-490b-9b39-40a396bc7ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:23.382352 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeeba57-42e2-490b-9b39-40a396bc7ce4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:23.393754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1225/2446] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:57:23.398033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41761e19-646a-400a-a179-fd1c88f56676 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:23.407814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41761e19-646a-400a-a179-fd1c88f56676 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:23.413709 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc3b59-2851-4ea3-8175-e7fdc269470a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:23.414295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc3b59-2851-4ea3-8175-e7fdc269470a admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.414571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc3b59-2851-4ea3-8175-e7fdc269470a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.414839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc3b59-2851-4ea3-8175-e7fdc269470a admin admin] PATH_INFO: `/v3/projects/2d01e78a9d67499bb2e5fbfa0958e0c8/users/f5102bca6ad64148bbf2760535d518f0/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.429761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc3b59-2851-4ea3-8175-e7fdc269470a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d01e78a9d67499bb2e5fbfa0958e0c8, user_id=f5102bca6ad64148bbf2760535d518f0, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:23.432490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc3b59-2851-4ea3-8175-e7fdc269470a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:23.445748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1222/2447] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:57:23 2022] PUT /identity/v3/projects/2d01e78a9d67499bb2e5fbfa0958e0c8/users/f5102bca6ad64148bbf2760535d518f0/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:57:23.449624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeba57-42e2-490b-9b39-40a396bc7ce4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:23.460122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeeba57-42e2-490b-9b39-40a396bc7ce4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:23.515003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc7f17-a51a-4c57-bff2-1cc7277e1825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:23.515651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7f17-a51a-4c57-bff2-1cc7277e1825 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.515854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7f17-a51a-4c57-bff2-1cc7277e1825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.516140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc7f17-a51a-4c57-bff2-1cc7277e1825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.516774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc7f17-a51a-4c57-bff2-1cc7277e1825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:23.518998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc7f17-a51a-4c57-bff2-1cc7277e1825 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:23.528685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1226/2448] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:57:23.532772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc3b59-2851-4ea3-8175-e7fdc269470a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:23.543745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc3b59-2851-4ea3-8175-e7fdc269470a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:23.548404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:23.549260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.549540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.549816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] PATH_INFO: `/v3/projects/2d01e78a9d67499bb2e5fbfa0958e0c8/users/f5102bca6ad64148bbf2760535d518f0/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.551716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d01e78a9d67499bb2e5fbfa0958e0c8, user_id=f5102bca6ad64148bbf2760535d518f0, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:23.554373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:23.565947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1223/2449] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:57:23 2022] PUT /identity/v3/projects/2d01e78a9d67499bb2e5fbfa0958e0c8/users/f5102bca6ad64148bbf2760535d518f0/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:57:23.576244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f307-1ab8-4b2f-b858-f8922a806389 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:23.576534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f307-1ab8-4b2f-b858-f8922a806389 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:23.576810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f307-1ab8-4b2f-b858-f8922a806389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:23.605033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3107f307-1ab8-4b2f-b858-f8922a806389 None None] MFA Rules not processed for user `f5102bca6ad64148bbf2760535d518f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:57:23.606245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3107f307-1ab8-4b2f-b858-f8922a806389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:23.676306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1227/2450] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:57:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:57:23.967386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:23.969193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:24.026303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40950f-df52-4600-a8c4-3ca8fcd84c84 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:24.029932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:24.030523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:24.030796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:24.031061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:24.039886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:24.041337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:24.051029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1224/2451] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:57:23 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:57:33.672693 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=67 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:57:34.046619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:57:35.770805 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f307-1ab8-4b2f-b858-f8922a806389 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:57:35.772233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3107f307-1ab8-4b2f-b858-f8922a806389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:35.852928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f307-1ab8-4b2f-b858-f8922a806389 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:57:35.857887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:57:35.858507 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:57:35.858780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:57:35.859159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:57:35.860886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:57:35.872578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:57:35.874127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:57:35.883730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1228/2452] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:57:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:57:45.879655 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=36 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:21.354818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:21.356272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:21.390166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634fa2d-79f4-4093-adfd-a4ab3ba212df service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:21.400051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:21.400716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:21.400997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:21.401266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:21.403068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:21.473601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:21.475555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:21.493917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1225/2453] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:21.541593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:21.561446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83246506-0404-4639-ad84-d3b23e466cc0 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:21.565018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:21.565601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:21.565873 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:21.566143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:21.567829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:21.646796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:21.648485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:21.667775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1229/2454] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:21.713517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:21.722155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee2fc2-28b0-4fbf-a957-abf571a36a80 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:21.725971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48011dd-11ea-492e-ad9a-d75f3866e4b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:21.726567 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48011dd-11ea-492e-ad9a-d75f3866e4b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:21.726845 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48011dd-11ea-492e-ad9a-d75f3866e4b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:21.727116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48011dd-11ea-492e-ad9a-d75f3866e4b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:21.728867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c48011dd-11ea-492e-ad9a-d75f3866e4b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:21.802431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48011dd-11ea-492e-ad9a-d75f3866e4b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:21.804511 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48011dd-11ea-492e-ad9a-d75f3866e4b8 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:21.823390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1226/2455] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:22.152469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:22.213001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73f490-a56e-4b39-ad6a-a8b193609945 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:22.217492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:22.218157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:22.218450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:22.218716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] PATH_INFO: `/v3/users/a811bd2dad9e4ec393a31541de13a912` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:22.219840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a811bd2dad9e4ec393a31541de13a912)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:22.221765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:22.267579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a811bd2dad9e4ec393a31541de13a912'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.277797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:22.280123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a811bd2dad9e4ec393a31541de13a912'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.281268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a811bd2dad9e4ec393a31541de13a912'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.289100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a811bd2dad9e4ec393a31541de13a912'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.296935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a811bd2dad9e4ec393a31541de13a912'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.302959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1230/2456] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:59:22 2022] DELETE /identity/v3/users/a811bd2dad9e4ec393a31541de13a912 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:59:22.308290 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774659ff-7f94-4461-85b0-fddd9867724f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:22.308591 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774659ff-7f94-4461-85b0-fddd9867724f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:22.308856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774659ff-7f94-4461-85b0-fddd9867724f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:22.327734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-774659ff-7f94-4461-85b0-fddd9867724f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:22.328948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-774659ff-7f94-4461-85b0-fddd9867724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:22.367632 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=100 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:22.388409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1227/2457] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:59:22.398375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:22.399773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:22.466580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9893e4-5a25-4a35-bc7b-96807f1fa4d5 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:22.470281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da3c19-f0a3-4c00-8c47-b584b20773a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:22.470879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3c19-f0a3-4c00-8c47-b584b20773a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:22.471158 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3c19-f0a3-4c00-8c47-b584b20773a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:22.471428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da3c19-f0a3-4c00-8c47-b584b20773a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:22.480936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da3c19-f0a3-4c00-8c47-b584b20773a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:22.482362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da3c19-f0a3-4c00-8c47-b584b20773a4 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:22.491723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1231/2458] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:22.757664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774659ff-7f94-4461-85b0-fddd9867724f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:22.759345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-774659ff-7f94-4461-85b0-fddd9867724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:22.771414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774659ff-7f94-4461-85b0-fddd9867724f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:22.777108 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:22.777778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:22.778059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:22.778331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] PATH_INFO: `/v3/projects/943727c0b4394cc78960ffd4cee20877` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:22.779503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=943727c0b4394cc78960ffd4cee20877)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:22.781463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:22.831341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '943727c0b4394cc78960ffd4cee20877', 'request_id': 'req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.839183 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '943727c0b4394cc78960ffd4cee20877', 'request_id': 'req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.845731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '943727c0b4394cc78960ffd4cee20877', 'request_id': 'req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.856634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] The token cache is being invalidate because project 943727c0b4394cc78960ffd4cee20877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:59:22.856923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.857555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.859037 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1228/2459] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:59:22 2022] DELETE /identity/v3/projects/943727c0b4394cc78960ffd4cee20877 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:59:22.863243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da3c19-f0a3-4c00-8c47-b584b20773a4 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:22.873107 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da3c19-f0a3-4c00-8c47-b584b20773a4 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:22.928775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:22.929392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:22.929670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:22.929940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] PATH_INFO: `/v3/users/f5102bca6ad64148bbf2760535d518f0` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:22.931052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5102bca6ad64148bbf2760535d518f0)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:22.932986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:22.981435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5102bca6ad64148bbf2760535d518f0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.993986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5102bca6ad64148bbf2760535d518f0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:22.994709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5102bca6ad64148bbf2760535d518f0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:23.001881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5102bca6ad64148bbf2760535d518f0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:23.003463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:23.017456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5102bca6ad64148bbf2760535d518f0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:23.024654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1232/2460] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:59:22 2022] DELETE /identity/v3/users/f5102bca6ad64148bbf2760535d518f0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:59:23.211523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:23.212991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:23.272752 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5564af-92f4-4bd5-a4f3-c7c1de3180fc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:23.306762 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:23.308434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:23.342067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23101d17-6d0b-4c9c-b1ef-b1e025fcfbc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:23.342713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23101d17-6d0b-4c9c-b1ef-b1e025fcfbc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.342995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23101d17-6d0b-4c9c-b1ef-b1e025fcfbc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.343269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23101d17-6d0b-4c9c-b1ef-b1e025fcfbc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.345078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-23101d17-6d0b-4c9c-b1ef-b1e025fcfbc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:23.375850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89461456-d8f8-4c6e-9b9a-8b0d38638bbf admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:23.380205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:23.380720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:23.381087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.381428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.381689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] PATH_INFO: `/v3/projects/2d01e78a9d67499bb2e5fbfa0958e0c8` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.382985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d01e78a9d67499bb2e5fbfa0958e0c8)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:23.385058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:23.413122 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23101d17-6d0b-4c9c-b1ef-b1e025fcfbc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:23.414831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23101d17-6d0b-4c9c-b1ef-b1e025fcfbc4 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:23.425138 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1229/2461] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:23.432626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d01e78a9d67499bb2e5fbfa0958e0c8', 'request_id': 'req-84ee942a-fb21-46a9-8a62-0832aea4e7c2'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:23.439367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d01e78a9d67499bb2e5fbfa0958e0c8', 'request_id': 'req-84ee942a-fb21-46a9-8a62-0832aea4e7c2'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:23.445067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d01e78a9d67499bb2e5fbfa0958e0c8', 'request_id': 'req-84ee942a-fb21-46a9-8a62-0832aea4e7c2'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:23.458892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] The token cache is being invalidate because project 2d01e78a9d67499bb2e5fbfa0958e0c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:59:23.459276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:23.460760 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-84ee942a-fb21-46a9-8a62-0832aea4e7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:23.462293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1233/2462] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:59:23 2022] DELETE /identity/v3/projects/2d01e78a9d67499bb2e5fbfa0958e0c8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:59:23.467172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49d6d9-48d1-4f7b-9c92-940abd55785e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.467528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49d6d9-48d1-4f7b-9c92-940abd55785e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.467806 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49d6d9-48d1-4f7b-9c92-940abd55785e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.486858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0b49d6d9-48d1-4f7b-9c92-940abd55785e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:23.488056 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0b49d6d9-48d1-4f7b-9c92-940abd55785e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:23.557219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1230/2463] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:59:23.558835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf13a3-5e45-498f-a289-dc80559b8d7e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.559122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf13a3-5e45-498f-a289-dc80559b8d7e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.559397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf13a3-5e45-498f-a289-dc80559b8d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.581335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d0bf13a3-5e45-498f-a289-dc80559b8d7e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:23.582560 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d0bf13a3-5e45-498f-a289-dc80559b8d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:23.590913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1234/2464] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:59:23.595133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49d6d9-48d1-4f7b-9c92-940abd55785e None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:23.605571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49d6d9-48d1-4f7b-9c92-940abd55785e None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:23.610342 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:23.610947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.611288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.611643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.612323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:59:23.612636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:23.614207 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:23.626815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1231/2465] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:59:23.631798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf13a3-5e45-498f-a289-dc80559b8d7e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:23.641435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf13a3-5e45-498f-a289-dc80559b8d7e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:23.646157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ea9cc-0560-4cd0-b821-96eff9d99ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:23.646778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ea9cc-0560-4cd0-b821-96eff9d99ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.647880 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ea9cc-0560-4cd0-b821-96eff9d99ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.648249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ea9cc-0560-4cd0-b821-96eff9d99ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.648928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ea9cc-0560-4cd0-b821-96eff9d99ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:23.650874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ea9cc-0560-4cd0-b821-96eff9d99ba4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:23.681302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1235/2466] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:59:23 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:59:23.685366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:23.696723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7fa3c-4434-4b62-97c4-ef3a71654b74 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:23.753072 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c51ca-de0b-483b-b027-45a44df52ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:23.753778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c51ca-de0b-483b-b027-45a44df52ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.753986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c51ca-de0b-483b-b027-45a44df52ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.754386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c51ca-de0b-483b-b027-45a44df52ce2 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.755239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c51ca-de0b-483b-b027-45a44df52ce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:23.757763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c51ca-de0b-483b-b027-45a44df52ce2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:23.784234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1232/2467] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:59:23 2022] POST /identity/v3/users => generated 305 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:59:23.788593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ea9cc-0560-4cd0-b821-96eff9d99ba4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:23.798972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ea9cc-0560-4cd0-b821-96eff9d99ba4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:23.803731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006689a2-2282-4696-8e57-877b0dfe7c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:23.804343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006689a2-2282-4696-8e57-877b0dfe7c43 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.804637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006689a2-2282-4696-8e57-877b0dfe7c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.804889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006689a2-2282-4696-8e57-877b0dfe7c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.805530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006689a2-2282-4696-8e57-877b0dfe7c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:23.807106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006689a2-2282-4696-8e57-877b0dfe7c43 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:23.819704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1236/2468] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:59:23.824043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c51ca-de0b-483b-b027-45a44df52ce2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:23.834153 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c51ca-de0b-483b-b027-45a44df52ce2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:23.840169 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a9978-092c-44eb-b0c7-1299b254bd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:23.840896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9978-092c-44eb-b0c7-1299b254bd60 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.841176 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9978-092c-44eb-b0c7-1299b254bd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.841444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9978-092c-44eb-b0c7-1299b254bd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.842082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a9978-092c-44eb-b0c7-1299b254bd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:23.843960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a9978-092c-44eb-b0c7-1299b254bd60 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:23.856008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1233/2469] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:59:23.860029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006689a2-2282-4696-8e57-877b0dfe7c43 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:23.870262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006689a2-2282-4696-8e57-877b0dfe7c43 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:23.875153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48c5a8-14d1-45e9-ab40-6292a0a81a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:23.875852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48c5a8-14d1-45e9-ab40-6292a0a81a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.876164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48c5a8-14d1-45e9-ab40-6292a0a81a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.876477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48c5a8-14d1-45e9-ab40-6292a0a81a8b admin admin] PATH_INFO: `/v3/projects/a2a22638fd964339965f6b51e8f3d73f/users/730fc071a42f4d0385a1b3a6d72b7781/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.894249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48c5a8-14d1-45e9-ab40-6292a0a81a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a22638fd964339965f6b51e8f3d73f, user_id=730fc071a42f4d0385a1b3a6d72b7781, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:23.899386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48c5a8-14d1-45e9-ab40-6292a0a81a8b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:23.912379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1237/2470] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:59:23 2022] PUT /identity/v3/projects/a2a22638fd964339965f6b51e8f3d73f/users/730fc071a42f4d0385a1b3a6d72b7781/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:59:23.924191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b2a2-0d30-46f5-9937-319081af0a09 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:23.924478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b2a2-0d30-46f5-9937-319081af0a09 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:23.924751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159b2a2-0d30-46f5-9937-319081af0a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:23.953713 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c159b2a2-0d30-46f5-9937-319081af0a09 None None] MFA Rules not processed for user `730fc071a42f4d0385a1b3a6d72b7781`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:23.954969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c159b2a2-0d30-46f5-9937-319081af0a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:23.982477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:24.025783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1234/2471] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:23 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:59:24.030596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48c5a8-14d1-45e9-ab40-6292a0a81a8b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:24.051828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:24.122414 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48c5a8-14d1-45e9-ab40-6292a0a81a8b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:24.127899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2d3c-9052-4eb0-86d2-308abe7af47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:24.128888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2d3c-9052-4eb0-86d2-308abe7af47c admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.129276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2d3c-9052-4eb0-86d2-308abe7af47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.130587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef2d3c-9052-4eb0-86d2-308abe7af47c admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.132388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef2d3c-9052-4eb0-86d2-308abe7af47c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:24.134399 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef2d3c-9052-4eb0-86d2-308abe7af47c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:24.174871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1238/2472] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:59:24 2022] POST /identity/v3/projects => generated 409 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:59:24.179642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159b2a2-0d30-46f5-9937-319081af0a09 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:24.190907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159b2a2-0d30-46f5-9937-319081af0a09 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:24.259481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b21883-a630-416c-9438-9d7cb45e47f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:24.260120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b21883-a630-416c-9438-9d7cb45e47f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.260404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b21883-a630-416c-9438-9d7cb45e47f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.260669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b21883-a630-416c-9438-9d7cb45e47f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.261422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b21883-a630-416c-9438-9d7cb45e47f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:24.263488 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b21883-a630-416c-9438-9d7cb45e47f2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:24.285556 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1235/2473] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:59:24 2022] POST /identity/v3/users => generated 311 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 16:59:24.290306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2d3c-9052-4eb0-86d2-308abe7af47c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:24.301028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef2d3c-9052-4eb0-86d2-308abe7af47c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:24.305846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be1f1-0989-417f-abb1-092ce7b7fe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:24.306450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be1f1-0989-417f-abb1-092ce7b7fe94 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.306728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be1f1-0989-417f-abb1-092ce7b7fe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.306997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be1f1-0989-417f-abb1-092ce7b7fe94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.307643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be1f1-0989-417f-abb1-092ce7b7fe94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:24.309279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be1f1-0989-417f-abb1-092ce7b7fe94 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:24.321490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1239/2474] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:59:24.325923 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b21883-a630-416c-9438-9d7cb45e47f2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:24.336986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b21883-a630-416c-9438-9d7cb45e47f2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:24.341894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6b559-5b46-4217-b87a-961e50e495b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:24.342635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6b559-5b46-4217-b87a-961e50e495b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.342982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6b559-5b46-4217-b87a-961e50e495b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.343334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6b559-5b46-4217-b87a-961e50e495b6 admin admin] PATH_INFO: `/v3/projects/e349069a28924731a55bf9f108ab338a/users/5f0ea3ac9db14553846d271cdb044d71/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.358792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6b559-5b46-4217-b87a-961e50e495b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e349069a28924731a55bf9f108ab338a, user_id=5f0ea3ac9db14553846d271cdb044d71, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:24.361768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6b559-5b46-4217-b87a-961e50e495b6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:24.374883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1236/2475] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:59:24 2022] PUT /identity/v3/projects/e349069a28924731a55bf9f108ab338a/users/5f0ea3ac9db14553846d271cdb044d71/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:59:24.379082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be1f1-0989-417f-abb1-092ce7b7fe94 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:24.389698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be1f1-0989-417f-abb1-092ce7b7fe94 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:24.447485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441772b7-d765-46aa-b15d-7758ea3ffe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:24.448088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441772b7-d765-46aa-b15d-7758ea3ffe85 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.448420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441772b7-d765-46aa-b15d-7758ea3ffe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.448691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441772b7-d765-46aa-b15d-7758ea3ffe85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.449336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441772b7-d765-46aa-b15d-7758ea3ffe85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:24.451078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441772b7-d765-46aa-b15d-7758ea3ffe85 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:24.465674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1240/2476] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:59:24.466815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6b559-5b46-4217-b87a-961e50e495b6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:24.478448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6b559-5b46-4217-b87a-961e50e495b6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:24.483168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:24.483822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.484191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.484408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] PATH_INFO: `/v3/projects/e349069a28924731a55bf9f108ab338a/users/5f0ea3ac9db14553846d271cdb044d71/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.486321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e349069a28924731a55bf9f108ab338a, user_id=5f0ea3ac9db14553846d271cdb044d71, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:24.489110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:24.501068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1237/2477] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:59:24 2022] PUT /identity/v3/projects/e349069a28924731a55bf9f108ab338a/users/5f0ea3ac9db14553846d271cdb044d71/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:59:24.512061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cee2-2c88-4f6c-a8f5-5ee15eeb0b13 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.513994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cee2-2c88-4f6c-a8f5-5ee15eeb0b13 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.513994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cee2-2c88-4f6c-a8f5-5ee15eeb0b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.546025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f2d5cee2-2c88-4f6c-a8f5-5ee15eeb0b13 None None] MFA Rules not processed for user `5f0ea3ac9db14553846d271cdb044d71`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:24.548041 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f2d5cee2-2c88-4f6c-a8f5-5ee15eeb0b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:24.635076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1241/2478] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:24 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:59:24.645989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:24.648619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:24.718538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb97bccd-cec4-412a-8118-aa0b76d2e135 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:24.722373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:24.723065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.723416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.723759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.734361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:24.735954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:24.746997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1238/2479] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:24 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:24.903503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5cee2-2c88-4f6c-a8f5-5ee15eeb0b13 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:24.912874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5cee2-2c88-4f6c-a8f5-5ee15eeb0b13 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:24.916565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:24.917234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:24.917575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:24.918147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:24.927544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:24.978522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:24.979968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:24.990305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1242/2480] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:24 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:26.618422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:26.619935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:26.646430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:26.688781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:26.690228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:26.712855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e3926-e10c-4c35-8da4-345fb1518cfe service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:26.718203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:26.718871 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:26.719259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:26.719613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:26.721520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:26.734584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:26.736393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:26.748077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1239/2481] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:26.767879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93298de4-c05e-4dc3-862c-a25405796673 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:26.772792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:26.773680 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:26.773964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:26.774233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] PATH_INFO: `/v3/users/60ee85a52c0244879f10e8943d20dfb8` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:26.775661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ee85a52c0244879f10e8943d20dfb8)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:26.777610 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:26.827064 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ee85a52c0244879f10e8943d20dfb8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:26.838780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ee85a52c0244879f10e8943d20dfb8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:26.839436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ee85a52c0244879f10e8943d20dfb8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:26.846144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ee85a52c0244879f10e8943d20dfb8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:26.853517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e485525-79a8-49f0-9292-1c2b16459e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ee85a52c0244879f10e8943d20dfb8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:26.859634 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1243/2482] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 16:59:26 2022] DELETE /identity/v3/users/60ee85a52c0244879f10e8943d20dfb8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 16:59:27.031084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.042832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5021df-b181-4e67-8af4-4238f868b532 service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.103362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.103994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.104303 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.104576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] PATH_INFO: `/v3/projects/fa83645c0fed4133b9eaf4c67e2bbad8` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.105699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa83645c0fed4133b9eaf4c67e2bbad8)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.107603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.153982 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa83645c0fed4133b9eaf4c67e2bbad8', 'request_id': 'req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:27.160469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa83645c0fed4133b9eaf4c67e2bbad8', 'request_id': 'req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:27.167446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa83645c0fed4133b9eaf4c67e2bbad8', 'request_id': 'req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:27.177561 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] The token cache is being invalidate because project fa83645c0fed4133b9eaf4c67e2bbad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 16:59:27.177844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:27.178506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d73c04ef-5ff2-46c1-8287-6baeed37f7b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 16:59:27.179997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1240/2483] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 16:59:27 2022] DELETE /identity/v3/projects/fa83645c0fed4133b9eaf4c67e2bbad8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:59:27.185621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742944a-5353-4037-b577-09cbc9bd497f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.185905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742944a-5353-4037-b577-09cbc9bd497f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.186177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742944a-5353-4037-b577-09cbc9bd497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.205665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-8742944a-5353-4037-b577-09cbc9bd497f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:27.206852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8742944a-5353-4037-b577-09cbc9bd497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:27.232454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:27.266764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1244/2484] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 16:59:27.275393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c6e09-1a37-41dd-8122-731f2d47e7f1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.275818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c6e09-1a37-41dd-8122-731f2d47e7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.276125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c6e09-1a37-41dd-8122-731f2d47e7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.295929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-247c6e09-1a37-41dd-8122-731f2d47e7f1 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:27.297141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-247c6e09-1a37-41dd-8122-731f2d47e7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:27.304901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1241/2485] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:59:27.309000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742944a-5353-4037-b577-09cbc9bd497f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.318430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742944a-5353-4037-b577-09cbc9bd497f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.323801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.324428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.324705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.324973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.325614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 16:59:27.325938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.327537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.339535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1245/2486] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 16:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 16:59:27.343641 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c6e09-1a37-41dd-8122-731f2d47e7f1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.352826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c6e09-1a37-41dd-8122-731f2d47e7f1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.357535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2323ed6-a4b7-4752-a29c-60a9ad231c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.358133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2323ed6-a4b7-4752-a29c-60a9ad231c37 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.358405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2323ed6-a4b7-4752-a29c-60a9ad231c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.358671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2323ed6-a4b7-4752-a29c-60a9ad231c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.359299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2323ed6-a4b7-4752-a29c-60a9ad231c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.361221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2323ed6-a4b7-4752-a29c-60a9ad231c37 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.387841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1242/2487] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:59:27 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:59:27.392477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.403245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165fbc3-719a-4d7e-b537-9dde2223470b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.463449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a806744-8098-4706-b57f-2634bd11ea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.464071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a806744-8098-4706-b57f-2634bd11ea80 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.464357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a806744-8098-4706-b57f-2634bd11ea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.464624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a806744-8098-4706-b57f-2634bd11ea80 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.465251 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a806744-8098-4706-b57f-2634bd11ea80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.467172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a806744-8098-4706-b57f-2634bd11ea80 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.488825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1246/2488] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:59:27 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:59:27.493354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2323ed6-a4b7-4752-a29c-60a9ad231c37 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.503624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2323ed6-a4b7-4752-a29c-60a9ad231c37 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.509389 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcef97-1a70-4502-bee1-3f6a8995500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.509978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcef97-1a70-4502-bee1-3f6a8995500f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.510254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcef97-1a70-4502-bee1-3f6a8995500f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.510524 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcef97-1a70-4502-bee1-3f6a8995500f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.511160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcef97-1a70-4502-bee1-3f6a8995500f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.512777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcef97-1a70-4502-bee1-3f6a8995500f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.524184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1243/2489] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 16:59:27.527911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a806744-8098-4706-b57f-2634bd11ea80 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.538269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a806744-8098-4706-b57f-2634bd11ea80 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.544699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a0602-6174-4c23-a3dc-9a118145ca52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.545301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a0602-6174-4c23-a3dc-9a118145ca52 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.545581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a0602-6174-4c23-a3dc-9a118145ca52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.545851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a0602-6174-4c23-a3dc-9a118145ca52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.546496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a0602-6174-4c23-a3dc-9a118145ca52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.548081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a0602-6174-4c23-a3dc-9a118145ca52 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.560830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1247/2490] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:59:27.564916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcef97-1a70-4502-bee1-3f6a8995500f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.571496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:27.578337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcef97-1a70-4502-bee1-3f6a8995500f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.586442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ff7d0-74f0-484d-bcfb-78c9e9981bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.587100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ff7d0-74f0-484d-bcfb-78c9e9981bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.587380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ff7d0-74f0-484d-bcfb-78c9e9981bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.587648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ff7d0-74f0-484d-bcfb-78c9e9981bf2 admin admin] PATH_INFO: `/v3/projects/2e8a72152aa74463a4d321827f18dd45/users/abef050ce49f40de82a0415535e31c85/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.603255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ff7d0-74f0-484d-bcfb-78c9e9981bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8a72152aa74463a4d321827f18dd45, user_id=abef050ce49f40de82a0415535e31c85, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.605922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ff7d0-74f0-484d-bcfb-78c9e9981bf2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.620384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1244/2491] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:59:27 2022] PUT /identity/v3/projects/2e8a72152aa74463a4d321827f18dd45/users/abef050ce49f40de82a0415535e31c85/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:59:27.632948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218617a7-86a1-41e3-b9be-9aee42734729 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.633236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218617a7-86a1-41e3-b9be-9aee42734729 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.633508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218617a7-86a1-41e3-b9be-9aee42734729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.662307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-218617a7-86a1-41e3-b9be-9aee42734729 None None] MFA Rules not processed for user `abef050ce49f40de82a0415535e31c85`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:27.664260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-218617a7-86a1-41e3-b9be-9aee42734729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:27.728738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1248/2492] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:59:27.733153 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ff7d0-74f0-484d-bcfb-78c9e9981bf2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.794950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ff7d0-74f0-484d-bcfb-78c9e9981bf2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.799648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741339f-04df-435f-b8ef-69fd9b606e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.800269 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741339f-04df-435f-b8ef-69fd9b606e98 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.800545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741339f-04df-435f-b8ef-69fd9b606e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.800813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741339f-04df-435f-b8ef-69fd9b606e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.801498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741339f-04df-435f-b8ef-69fd9b606e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.803421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741339f-04df-435f-b8ef-69fd9b606e98 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.829274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1245/2493] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 16:59:27 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:59:27.833256 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218617a7-86a1-41e3-b9be-9aee42734729 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.844116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218617a7-86a1-41e3-b9be-9aee42734729 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.867736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:27.899539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f32ed-3482-41a1-88e1-4016c1cbf5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.900157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f32ed-3482-41a1-88e1-4016c1cbf5bd admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.900437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f32ed-3482-41a1-88e1-4016c1cbf5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.900703 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f32ed-3482-41a1-88e1-4016c1cbf5bd admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.901317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f32ed-3482-41a1-88e1-4016c1cbf5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.903206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f32ed-3482-41a1-88e1-4016c1cbf5bd admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.924248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1249/2494] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 16:59:27 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 16:59:27.928239 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741339f-04df-435f-b8ef-69fd9b606e98 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.938683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741339f-04df-435f-b8ef-69fd9b606e98 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.943305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96411405-5753-4be6-9f74-51d2c5ff070a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.943939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411405-5753-4be6-9f74-51d2c5ff070a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.944266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411405-5753-4be6-9f74-51d2c5ff070a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.944537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96411405-5753-4be6-9f74-51d2c5ff070a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.945175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96411405-5753-4be6-9f74-51d2c5ff070a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.947061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96411405-5753-4be6-9f74-51d2c5ff070a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:27.957438 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1246/2495] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:59:27.961987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f32ed-3482-41a1-88e1-4016c1cbf5bd admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:27.971812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f32ed-3482-41a1-88e1-4016c1cbf5bd admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:27.977214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03e34-a3bc-4da6-b3b2-385918b4363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:27.977863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03e34-a3bc-4da6-b3b2-385918b4363f admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:27.978141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03e34-a3bc-4da6-b3b2-385918b4363f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:27.978437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03e34-a3bc-4da6-b3b2-385918b4363f admin admin] PATH_INFO: `/v3/projects/aee1921ffa4848b7922b2e9fb26102c5/users/75ecce90109746978e2ec70b43263113/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:27.993191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d03e34-a3bc-4da6-b3b2-385918b4363f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee1921ffa4848b7922b2e9fb26102c5, user_id=75ecce90109746978e2ec70b43263113, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:27.995802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d03e34-a3bc-4da6-b3b2-385918b4363f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:28.009021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1250/2496] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:59:27 2022] PUT /identity/v3/projects/aee1921ffa4848b7922b2e9fb26102c5/users/75ecce90109746978e2ec70b43263113/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:59:28.012715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96411405-5753-4be6-9f74-51d2c5ff070a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:28.023581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96411405-5753-4be6-9f74-51d2c5ff070a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:28.087361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208543a8-2a9a-4247-afad-47fdceef820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:28.087968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208543a8-2a9a-4247-afad-47fdceef820c admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:28.088284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208543a8-2a9a-4247-afad-47fdceef820c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:28.088554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208543a8-2a9a-4247-afad-47fdceef820c admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:28.089189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208543a8-2a9a-4247-afad-47fdceef820c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:28.090769 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208543a8-2a9a-4247-afad-47fdceef820c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:28.100777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1247/2497] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 16:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 16:59:28.105154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03e34-a3bc-4da6-b3b2-385918b4363f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:28.116249 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03e34-a3bc-4da6-b3b2-385918b4363f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:28.122031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:28.122683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:28.122962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:28.123232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] PATH_INFO: `/v3/projects/aee1921ffa4848b7922b2e9fb26102c5/users/75ecce90109746978e2ec70b43263113/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:28.125233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee1921ffa4848b7922b2e9fb26102c5, user_id=75ecce90109746978e2ec70b43263113, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:28.127872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:28.139054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1251/2498] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 16:59:28 2022] PUT /identity/v3/projects/aee1921ffa4848b7922b2e9fb26102c5/users/75ecce90109746978e2ec70b43263113/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 16:59:28.149973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05336428-be1f-4640-92bc-8b9f2f824ff5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:28.150256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05336428-be1f-4640-92bc-8b9f2f824ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:28.150529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05336428-be1f-4640-92bc-8b9f2f824ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:28.177665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-05336428-be1f-4640-92bc-8b9f2f824ff5 None None] MFA Rules not processed for user `75ecce90109746978e2ec70b43263113`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 16:59:28.179054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-05336428-be1f-4640-92bc-8b9f2f824ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:28.217618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:28.247765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1248/2499] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 16:59:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 16:59:28.259948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:28.261662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:28.319528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f8ff-df3b-4321-94b2-07b6d1393f70 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:28.323431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:28.324028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:28.324337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:28.324605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:28.333581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:28.335766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:28.345231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1252/2500] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:28.417155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05336428-be1f-4640-92bc-8b9f2f824ff5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:28.426763 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05336428-be1f-4640-92bc-8b9f2f824ff5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:28.431252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:28.431847 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:28.432146 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:28.432420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:28.489560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:28.491521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:28.500280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1249/2501] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:33.506791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 16:59:33.508285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:33.564692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66450c67-801f-46e2-8a59-0a73151bd5f7 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 16:59:33.568373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 16:59:33.569601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 16:59:33.569877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 16:59:33.570148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 16:59:33.572165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 16:59:33.582167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 16:59:33.583543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 16:59:33.592740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1253/2502] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 16:59:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 16:59:38.496322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=40 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 16:59:43.589139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:01:24.270521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:01:24.271955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:01:24.287530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5810f-aa32-43dc-9164-35c20323ef77 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:01:24.292614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:01:24.293584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:01:24.293938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:01:24.294277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:01:24.296330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:01:24.379733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:01:24.381233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:01:24.402574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1250/2503] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:01:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:01:25.729374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:01:25.738456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b4878-8a80-4d57-9fa1-2b59966a1bcb service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:01:25.742524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:01:25.743359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:01:25.743828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:01:25.744325 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:01:25.747048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:01:25.762749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:01:25.764260 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:01:25.776796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1254/2504] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:01:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:01:25.880174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:01:25.881791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:01:25.906719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21e0d-78d9-42b7-80f9-5587d02271ee service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:01:25.914819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:01:25.915429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:01:25.915795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:01:25.916011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:01:25.918510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:01:25.933984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:01:25.935596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:01:25.947640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1251/2505] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:01:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:01:35.770969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=6 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:01:35.943497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=47 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:02:37.818932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:02:37.820404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:02:37.908371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a205c9-2ea9-4824-bba0-159727e1776f service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:02:37.913280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:02:37.913874 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:02:37.914149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:02:37.914415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:02:37.916142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:02:37.927185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:02:37.928775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:02:37.938969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1255/2506] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:02:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:02:47.934296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=41 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:12.527652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:12.529189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:12.564842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa5ecd-85a1-465f-8490-839f4d3bc339 service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:12.569277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:12.569893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:12.570166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:12.570433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:12.572253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:12.667575 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:12.669100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:12.690327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1252/2507] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:12.957651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:12.969165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e465-af93-4f6c-9666-2c70c75dbb37 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:12.975772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699569a-5a39-4922-81a5-c9bba5d82368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:12.976734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699569a-5a39-4922-81a5-c9bba5d82368 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:12.977305 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699569a-5a39-4922-81a5-c9bba5d82368 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:12.977807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699569a-5a39-4922-81a5-c9bba5d82368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:12.980054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9699569a-5a39-4922-81a5-c9bba5d82368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:13.086553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699569a-5a39-4922-81a5-c9bba5d82368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:13.088181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699569a-5a39-4922-81a5-c9bba5d82368 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:13.109717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1256/2508] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:03:22.687052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=49 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:23.106885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=43 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:24.130859 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:24.132407 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:24.222231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faea7c4-7df0-46bd-b92d-c7fd44e81974 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:24.227039 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:24.227681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:24.227959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:24.228410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] PATH_INFO: `/v3/users/730fc071a42f4d0385a1b3a6d72b7781` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:24.229529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=730fc071a42f4d0385a1b3a6d72b7781)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:24.231467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:24.288732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730fc071a42f4d0385a1b3a6d72b7781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.296993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730fc071a42f4d0385a1b3a6d72b7781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.298250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730fc071a42f4d0385a1b3a6d72b7781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.315995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730fc071a42f4d0385a1b3a6d72b7781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.316955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730fc071a42f4d0385a1b3a6d72b7781'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.332424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1253/2509] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:24 2022] DELETE /identity/v3/users/730fc071a42f4d0385a1b3a6d72b7781 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:24.337492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c577cc-15f2-4d9e-9f11-ee07a9523df4 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:24.337965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c577cc-15f2-4d9e-9f11-ee07a9523df4 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:24.338224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c577cc-15f2-4d9e-9f11-ee07a9523df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:24.359332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-50c577cc-15f2-4d9e-9f11-ee07a9523df4 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:24.360788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-50c577cc-15f2-4d9e-9f11-ee07a9523df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:24.439094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1257/2510] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:24.451196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:24.452856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:24.516850 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8e961b-8e45-4741-a628-67c0e769ec88 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:24.521181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bc169-6b67-46b6-9fc1-5fe60e58869c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:24.521788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bc169-6b67-46b6-9fc1-5fe60e58869c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:24.522060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bc169-6b67-46b6-9fc1-5fe60e58869c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:24.522327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bc169-6b67-46b6-9fc1-5fe60e58869c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:24.531638 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bc169-6b67-46b6-9fc1-5fe60e58869c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:24.533127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bc169-6b67-46b6-9fc1-5fe60e58869c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:24.543469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1254/2511] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:24.842227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c577cc-15f2-4d9e-9f11-ee07a9523df4 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:24.843901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-50c577cc-15f2-4d9e-9f11-ee07a9523df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:24.859089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c577cc-15f2-4d9e-9f11-ee07a9523df4 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:24.866846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:24.867588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:24.867864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:24.868413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] PATH_INFO: `/v3/projects/a2a22638fd964339965f6b51e8f3d73f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:24.869832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a22638fd964339965f6b51e8f3d73f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:24.872375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:24.929536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a22638fd964339965f6b51e8f3d73f', 'request_id': 'req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.936192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a22638fd964339965f6b51e8f3d73f', 'request_id': 'req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.942614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a22638fd964339965f6b51e8f3d73f', 'request_id': 'req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.955409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] The token cache is being invalidate because project a2a22638fd964339965f6b51e8f3d73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:24.956208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.956845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:24.958587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1258/2512] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:24 2022] DELETE /identity/v3/projects/a2a22638fd964339965f6b51e8f3d73f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:24.962312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bc169-6b67-46b6-9fc1-5fe60e58869c service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:24.972605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bc169-6b67-46b6-9fc1-5fe60e58869c service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:24.991272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:25.035062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:25.035062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:25.035062 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:25.035451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] PATH_INFO: `/v3/users/5f0ea3ac9db14553846d271cdb044d71` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:25.036509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0ea3ac9db14553846d271cdb044d71)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:25.038422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:25.105286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0ea3ac9db14553846d271cdb044d71'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.114467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0ea3ac9db14553846d271cdb044d71'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.115117 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0ea3ac9db14553846d271cdb044d71'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.130696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0ea3ac9db14553846d271cdb044d71'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.132161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3edadc7c-5d21-444f-b40c-16b0ff38130b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0ea3ac9db14553846d271cdb044d71'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.148144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1255/2513] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:24 2022] DELETE /identity/v3/users/5f0ea3ac9db14553846d271cdb044d71 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:25.499707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:25.502013 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:25.519436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d50e7-3054-422c-96eb-c51fc42dfe1c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:25.591082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:25.591639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:25.591919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:25.592217 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] PATH_INFO: `/v3/projects/e349069a28924731a55bf9f108ab338a` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:25.593345 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e349069a28924731a55bf9f108ab338a)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:25.595394 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:25.635536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:25.654358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e349069a28924731a55bf9f108ab338a', 'request_id': 'req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.661826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e349069a28924731a55bf9f108ab338a', 'request_id': 'req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.667749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e349069a28924731a55bf9f108ab338a', 'request_id': 'req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.688184 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] The token cache is being invalidate because project e349069a28924731a55bf9f108ab338a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:25.688534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.689195 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-87528e36-3ceb-4b2f-be5a-a0e3ecf08848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:25.691415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1259/2514] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:25 2022] DELETE /identity/v3/projects/e349069a28924731a55bf9f108ab338a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:25.696285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a1e8-d46b-402f-8eb1-b6cb54a5b2d6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:25.696689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a1e8-d46b-402f-8eb1-b6cb54a5b2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:25.697157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a1e8-d46b-402f-8eb1-b6cb54a5b2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:25.727479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0f34a1e8-d46b-402f-8eb1-b6cb54a5b2d6 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:25.728855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0f34a1e8-d46b-402f-8eb1-b6cb54a5b2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:25.820767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1256/2515] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:25.828810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798f1d4-77e5-464a-a779-a29313bc8aec None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:25.829462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798f1d4-77e5-464a-a779-a29313bc8aec None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:25.829462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798f1d4-77e5-464a-a779-a29313bc8aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:25.862928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c798f1d4-77e5-464a-a779-a29313bc8aec None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:25.864415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c798f1d4-77e5-464a-a779-a29313bc8aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:25.873232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1260/2516] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:25.877744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34a1e8-d46b-402f-8eb1-b6cb54a5b2d6 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:25.887883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34a1e8-d46b-402f-8eb1-b6cb54a5b2d6 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:25.898525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:25.899174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:25.900277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:25.900537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:25.901553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:03:25.901881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:25.903699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:25.917322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1257/2517] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:03:25.921591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798f1d4-77e5-464a-a779-a29313bc8aec None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:25.932755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798f1d4-77e5-464a-a779-a29313bc8aec None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:25.938061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2b2cc-5e65-40b2-9817-32ef74b6a7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:25.938309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2b2cc-5e65-40b2-9817-32ef74b6a7ef admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:25.938613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2b2cc-5e65-40b2-9817-32ef74b6a7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:25.939215 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2b2cc-5e65-40b2-9817-32ef74b6a7ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:25.939544 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2b2cc-5e65-40b2-9817-32ef74b6a7ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:25.941926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be2b2cc-5e65-40b2-9817-32ef74b6a7ef admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:25.976724 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1261/2518] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:25 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:03:25.979650 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:25.992472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e2726-6084-4d2c-afe9-a783260a7a57 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.023973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:26.060773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a18cf9-71d7-4263-b624-a729370a5695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.061386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18cf9-71d7-4263-b624-a729370a5695 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.061668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18cf9-71d7-4263-b624-a729370a5695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.061939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18cf9-71d7-4263-b624-a729370a5695 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.062566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a18cf9-71d7-4263-b624-a729370a5695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.064513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a18cf9-71d7-4263-b624-a729370a5695 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.091602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1258/2519] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:25 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:03:26.092871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2b2cc-5e65-40b2-9817-32ef74b6a7ef admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.103451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2b2cc-5e65-40b2-9817-32ef74b6a7ef admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.108498 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f117a8-d39c-4a3c-8948-eefda7dc6c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.109239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f117a8-d39c-4a3c-8948-eefda7dc6c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.109546 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f117a8-d39c-4a3c-8948-eefda7dc6c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.109823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f117a8-d39c-4a3c-8948-eefda7dc6c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.110540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f117a8-d39c-4a3c-8948-eefda7dc6c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.112484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f117a8-d39c-4a3c-8948-eefda7dc6c38 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.123029 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1262/2520] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:26.127776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a18cf9-71d7-4263-b624-a729370a5695 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.138599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a18cf9-71d7-4263-b624-a729370a5695 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.145419 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e0cf2-fae3-46f6-8160-6e185a71c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.146036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0cf2-fae3-46f6-8160-6e185a71c5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.146311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0cf2-fae3-46f6-8160-6e185a71c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.146930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e0cf2-fae3-46f6-8160-6e185a71c5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.147536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e0cf2-fae3-46f6-8160-6e185a71c5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.149347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e0cf2-fae3-46f6-8160-6e185a71c5d4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.160429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1259/2521] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:26.166079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f117a8-d39c-4a3c-8948-eefda7dc6c38 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.177447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f117a8-d39c-4a3c-8948-eefda7dc6c38 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.182533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8092e06-451d-4e43-b728-15154ac2ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.183384 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8092e06-451d-4e43-b728-15154ac2ffe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.183768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8092e06-451d-4e43-b728-15154ac2ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.187131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8092e06-451d-4e43-b728-15154ac2ffe0 admin admin] PATH_INFO: `/v3/projects/5931b48f761a4e3ba42e1c67e74dc026/users/5313b6698e014f05af21be5c203e227b/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.201996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8092e06-451d-4e43-b728-15154ac2ffe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5931b48f761a4e3ba42e1c67e74dc026, user_id=5313b6698e014f05af21be5c203e227b, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.204696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8092e06-451d-4e43-b728-15154ac2ffe0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.218477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1263/2522] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:26 2022] PUT /identity/v3/projects/5931b48f761a4e3ba42e1c67e74dc026/users/5313b6698e014f05af21be5c203e227b/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:26.288329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60083cf-93ca-4a61-bd0a-991aa9dc638c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.288719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60083cf-93ca-4a61-bd0a-991aa9dc638c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.289086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60083cf-93ca-4a61-bd0a-991aa9dc638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.317385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d60083cf-93ca-4a61-bd0a-991aa9dc638c None None] MFA Rules not processed for user `5313b6698e014f05af21be5c203e227b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:26.318603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d60083cf-93ca-4a61-bd0a-991aa9dc638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:26.385252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1260/2523] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:26.389806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8092e06-451d-4e43-b728-15154ac2ffe0 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.454654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8092e06-451d-4e43-b728-15154ac2ffe0 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.460776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57942-7438-4998-9377-8123ebcd54db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.461428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57942-7438-4998-9377-8123ebcd54db admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.461772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57942-7438-4998-9377-8123ebcd54db admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.462059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c57942-7438-4998-9377-8123ebcd54db admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.462738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c57942-7438-4998-9377-8123ebcd54db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.464893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c57942-7438-4998-9377-8123ebcd54db admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.493848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1264/2524] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:26 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:26.498012 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60083cf-93ca-4a61-bd0a-991aa9dc638c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.509491 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60083cf-93ca-4a61-bd0a-991aa9dc638c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.568715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967bd9a-aa14-4006-a81e-91de47849932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.569320 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967bd9a-aa14-4006-a81e-91de47849932 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.569595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967bd9a-aa14-4006-a81e-91de47849932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.569910 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967bd9a-aa14-4006-a81e-91de47849932 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.570554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967bd9a-aa14-4006-a81e-91de47849932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.572629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967bd9a-aa14-4006-a81e-91de47849932 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.594373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1261/2525] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:26 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:03:26.598596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57942-7438-4998-9377-8123ebcd54db admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.609118 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c57942-7438-4998-9377-8123ebcd54db admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.614814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062bbb5-9a41-426c-b093-1d618a7bd679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.615451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062bbb5-9a41-426c-b093-1d618a7bd679 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.615732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062bbb5-9a41-426c-b093-1d618a7bd679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.616006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062bbb5-9a41-426c-b093-1d618a7bd679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.616730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062bbb5-9a41-426c-b093-1d618a7bd679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.618416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062bbb5-9a41-426c-b093-1d618a7bd679 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.630430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1265/2526] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:03:26.634832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967bd9a-aa14-4006-a81e-91de47849932 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.645436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967bd9a-aa14-4006-a81e-91de47849932 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.650518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72462625-00fc-4571-b0bf-b05687824abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.651256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72462625-00fc-4571-b0bf-b05687824abb admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.651566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72462625-00fc-4571-b0bf-b05687824abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.651840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72462625-00fc-4571-b0bf-b05687824abb admin admin] PATH_INFO: `/v3/projects/209953b4728543039b138362a92ac130/users/bd1df9bbb2174192988710138a48950c/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.668532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72462625-00fc-4571-b0bf-b05687824abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=209953b4728543039b138362a92ac130, user_id=bd1df9bbb2174192988710138a48950c, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.671421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72462625-00fc-4571-b0bf-b05687824abb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.684564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1262/2527] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:26 2022] PUT /identity/v3/projects/209953b4728543039b138362a92ac130/users/bd1df9bbb2174192988710138a48950c/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:26.688678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062bbb5-9a41-426c-b093-1d618a7bd679 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.700851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062bbb5-9a41-426c-b093-1d618a7bd679 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.731502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:26.780917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6936ad-f86b-4ec9-ba92-f20dec0a0f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.781534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6936ad-f86b-4ec9-ba92-f20dec0a0f3f admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.781811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6936ad-f86b-4ec9-ba92-f20dec0a0f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.782085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6936ad-f86b-4ec9-ba92-f20dec0a0f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.782727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6936ad-f86b-4ec9-ba92-f20dec0a0f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.785717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6936ad-f86b-4ec9-ba92-f20dec0a0f3f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.796326 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1266/2528] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:26.801380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72462625-00fc-4571-b0bf-b05687824abb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.807135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:26.814035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72462625-00fc-4571-b0bf-b05687824abb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:26.820887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:26.821550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.821852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.822118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] PATH_INFO: `/v3/projects/209953b4728543039b138362a92ac130/users/bd1df9bbb2174192988710138a48950c/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.824050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209953b4728543039b138362a92ac130, user_id=bd1df9bbb2174192988710138a48950c, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:26.827029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:26.837981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1263/2529] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:26 2022] PUT /identity/v3/projects/209953b4728543039b138362a92ac130/users/bd1df9bbb2174192988710138a48950c/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:03:26.849096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9062d-da7a-4d5e-9454-c5bfc9c01c98 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:26.849470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9062d-da7a-4d5e-9454-c5bfc9c01c98 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:26.849823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9062d-da7a-4d5e-9454-c5bfc9c01c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:26.886792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ebf9062d-da7a-4d5e-9454-c5bfc9c01c98 None None] MFA Rules not processed for user `bd1df9bbb2174192988710138a48950c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:26.888186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ebf9062d-da7a-4d5e-9454-c5bfc9c01c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:26.968674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1267/2530] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:26.981678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:26.983392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:27.090785 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8269304-05fe-44ce-96ad-cd2c2f3e7d37 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:27.094663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58497ae-0790-4982-9d65-541f3f5e07ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:27.095277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58497ae-0790-4982-9d65-541f3f5e07ce service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:27.095541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58497ae-0790-4982-9d65-541f3f5e07ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:27.095807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58497ae-0790-4982-9d65-541f3f5e07ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:27.105634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58497ae-0790-4982-9d65-541f3f5e07ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:27.107098 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58497ae-0790-4982-9d65-541f3f5e07ce service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:27.117630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1264/2531] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:27.978379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9062d-da7a-4d5e-9454-c5bfc9c01c98 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:28.051719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9062d-da7a-4d5e-9454-c5bfc9c01c98 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:28.057709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:28.058350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:28.058626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:28.058897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] PATH_INFO: `/v3/users/5313b6698e014f05af21be5c203e227b` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:28.060026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5313b6698e014f05af21be5c203e227b)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:28.061998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:28.118280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5313b6698e014f05af21be5c203e227b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.119457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:28.136529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5313b6698e014f05af21be5c203e227b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.137279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5313b6698e014f05af21be5c203e227b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.145533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5313b6698e014f05af21be5c203e227b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.153073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5313b6698e014f05af21be5c203e227b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.161290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1268/2532] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:27 2022] DELETE /identity/v3/users/5313b6698e014f05af21be5c203e227b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:03:28.164862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d0ba6-1ea3-46ec-a71c-85d54afc0106 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:28.165386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d0ba6-1ea3-46ec-a71c-85d54afc0106 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:28.165494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d0ba6-1ea3-46ec-a71c-85d54afc0106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:28.186293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0b4d0ba6-1ea3-46ec-a71c-85d54afc0106 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:28.187496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0b4d0ba6-1ea3-46ec-a71c-85d54afc0106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:28.248677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1265/2533] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:28.259239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:28.260699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:28.322437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88036b9-9135-4eba-be7b-e5b4b058e871 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:28.326738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998e161-9008-4cc3-9f1d-8de570613b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:28.327440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998e161-9008-4cc3-9f1d-8de570613b40 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:28.327717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998e161-9008-4cc3-9f1d-8de570613b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:28.327985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998e161-9008-4cc3-9f1d-8de570613b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:28.341075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998e161-9008-4cc3-9f1d-8de570613b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:28.342601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998e161-9008-4cc3-9f1d-8de570613b40 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:28.353423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1269/2534] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:28.647588 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d0ba6-1ea3-46ec-a71c-85d54afc0106 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:28.649337 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0b4d0ba6-1ea3-46ec-a71c-85d54afc0106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:28.662023 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d0ba6-1ea3-46ec-a71c-85d54afc0106 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:28.666834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:28.667601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:28.667951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:28.668382 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] PATH_INFO: `/v3/projects/5931b48f761a4e3ba42e1c67e74dc026` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:28.669607 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5931b48f761a4e3ba42e1c67e74dc026)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:28.671652 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:28.720251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5931b48f761a4e3ba42e1c67e74dc026', 'request_id': 'req-3ae36da4-8ff3-481b-8ce0-45807c8f979c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.727912 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5931b48f761a4e3ba42e1c67e74dc026', 'request_id': 'req-3ae36da4-8ff3-481b-8ce0-45807c8f979c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.734410 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5931b48f761a4e3ba42e1c67e74dc026', 'request_id': 'req-3ae36da4-8ff3-481b-8ce0-45807c8f979c'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.747408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] The token cache is being invalidate because project 5931b48f761a4e3ba42e1c67e74dc026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:28.747664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.748417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.750031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1266/2535] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:28 2022] DELETE /identity/v3/projects/5931b48f761a4e3ba42e1c67e74dc026 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:28.754373 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998e161-9008-4cc3-9f1d-8de570613b40 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:28.764308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998e161-9008-4cc3-9f1d-8de570613b40 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:28.839912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:28.840576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:28.840869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:28.841253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] PATH_INFO: `/v3/users/bd1df9bbb2174192988710138a48950c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:28.842672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd1df9bbb2174192988710138a48950c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:28.844572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:28.900934 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd1df9bbb2174192988710138a48950c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.916012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd1df9bbb2174192988710138a48950c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.916800 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd1df9bbb2174192988710138a48950c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.922252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:28.925802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd1df9bbb2174192988710138a48950c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.937516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd1df9bbb2174192988710138a48950c'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:28.945085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1270/2536] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:28 2022] DELETE /identity/v3/users/bd1df9bbb2174192988710138a48950c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:03:29.259644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.262050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:29.262636 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.264374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:29.274824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:29.280499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae36da4-8ff3-481b-8ce0-45807c8f979c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:29.333292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126938c1-e5b7-4067-bef8-ae16a0d79940 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:29.337357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a6fc7-4377-4203-b9c2-0b41b61ea09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:29.338245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a6fc7-4377-4203-b9c2-0b41b61ea09b service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.338689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a6fc7-4377-4203-b9c2-0b41b61ea09b service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.339122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a6fc7-4377-4203-b9c2-0b41b61ea09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.341480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1b6a6fc7-4377-4203-b9c2-0b41b61ea09b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:29.363127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:29.363898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.364212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.364490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] PATH_INFO: `/v3/projects/209953b4728543039b138362a92ac130` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.365729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209953b4728543039b138362a92ac130)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:29.367860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:29.421354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a6fc7-4377-4203-b9c2-0b41b61ea09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:29.423418 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a6fc7-4377-4203-b9c2-0b41b61ea09b service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:29.437897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209953b4728543039b138362a92ac130', 'request_id': 'req-42938c00-3be9-4799-b154-4cae8bbcff23'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:29.438428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1271/2537] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:03:29.448658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209953b4728543039b138362a92ac130', 'request_id': 'req-42938c00-3be9-4799-b154-4cae8bbcff23'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:29.459198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209953b4728543039b138362a92ac130', 'request_id': 'req-42938c00-3be9-4799-b154-4cae8bbcff23'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:29.470275 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] The token cache is being invalidate because project 209953b4728543039b138362a92ac130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:29.470647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:29.472115 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-42938c00-3be9-4799-b154-4cae8bbcff23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:29.473689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1267/2538] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:29 2022] DELETE /identity/v3/projects/209953b4728543039b138362a92ac130 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:29.479524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6ce3-808b-4bff-81ac-3f0118345f4a None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.479813 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6ce3-808b-4bff-81ac-3f0118345f4a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.480092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6ce3-808b-4bff-81ac-3f0118345f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.501337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4b8d6ce3-808b-4bff-81ac-3f0118345f4a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:29.502537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4b8d6ce3-808b-4bff-81ac-3f0118345f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:29.574363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1272/2539] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:03:29.583252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba23a8-33fb-489b-80f6-4378c9051a9b None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.583530 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba23a8-33fb-489b-80f6-4378c9051a9b None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.583793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba23a8-33fb-489b-80f6-4378c9051a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.604672 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2bba23a8-33fb-489b-80f6-4378c9051a9b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:29.605896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2bba23a8-33fb-489b-80f6-4378c9051a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:29.614085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1268/2540] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:29.618464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6ce3-808b-4bff-81ac-3f0118345f4a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.628928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6ce3-808b-4bff-81ac-3f0118345f4a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:29.633619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:29.634232 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.634511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.634922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.635674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:03:29.635976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:29.637769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:29.652148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1273/2541] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:03:29.656408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba23a8-33fb-489b-80f6-4378c9051a9b None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.667180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba23a8-33fb-489b-80f6-4378c9051a9b None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:29.674986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4b8df-6335-486b-89b2-d2a3dc314352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:29.675749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4b8df-6335-486b-89b2-d2a3dc314352 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.676092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4b8df-6335-486b-89b2-d2a3dc314352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.676526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4b8df-6335-486b-89b2-d2a3dc314352 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.677241 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4b8df-6335-486b-89b2-d2a3dc314352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:29.679266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4b8df-6335-486b-89b2-d2a3dc314352 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:29.712145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1269/2542] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:29 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:29.716030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.728537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0b99a-9f28-46ea-8f5c-0a299adab04c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:29.755810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:29.799320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819e90a-aa0b-4775-a0ed-47edd8744992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:29.799925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819e90a-aa0b-4775-a0ed-47edd8744992 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.800245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819e90a-aa0b-4775-a0ed-47edd8744992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.800516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819e90a-aa0b-4775-a0ed-47edd8744992 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.802084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819e90a-aa0b-4775-a0ed-47edd8744992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:29.803068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819e90a-aa0b-4775-a0ed-47edd8744992 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:29.828487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1274/2543] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:29 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:29.829729 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4b8df-6335-486b-89b2-d2a3dc314352 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.841196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4b8df-6335-486b-89b2-d2a3dc314352 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:29.846657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba7af-3342-4ffe-9bb8-26c18a4d0e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:29.847332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba7af-3342-4ffe-9bb8-26c18a4d0e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.847614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba7af-3342-4ffe-9bb8-26c18a4d0e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.847894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba7af-3342-4ffe-9bb8-26c18a4d0e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.848857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba7af-3342-4ffe-9bb8-26c18a4d0e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:29.850777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba7af-3342-4ffe-9bb8-26c18a4d0e97 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:29.861577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1270/2544] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:29.865664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819e90a-aa0b-4775-a0ed-47edd8744992 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.875971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819e90a-aa0b-4775-a0ed-47edd8744992 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:29.881856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79154a45-26fc-4a23-9042-c06fe33511fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:29.882510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79154a45-26fc-4a23-9042-c06fe33511fc admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.882790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79154a45-26fc-4a23-9042-c06fe33511fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.883068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79154a45-26fc-4a23-9042-c06fe33511fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.883715 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79154a45-26fc-4a23-9042-c06fe33511fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:29.885379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79154a45-26fc-4a23-9042-c06fe33511fc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:29.901557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1275/2545] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:29.902786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba7af-3342-4ffe-9bb8-26c18a4d0e97 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.913523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba7af-3342-4ffe-9bb8-26c18a4d0e97 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:29.918172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:29.918861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.919262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.919541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] PATH_INFO: `/v3/projects/47e4979b4eee43989590f7e2fbc50204/users/e574095b22114893b590b94d83439574/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.934461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e4979b4eee43989590f7e2fbc50204, user_id=e574095b22114893b590b94d83439574, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:29.937134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:29.958863 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1271/2546] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:29 2022] PUT /identity/v3/projects/47e4979b4eee43989590f7e2fbc50204/users/e574095b22114893b590b94d83439574/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:29.971471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423bcd-49b5-4b21-9b49-df2bac741a7e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:29.971820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423bcd-49b5-4b21-9b49-df2bac741a7e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:29.972116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87423bcd-49b5-4b21-9b49-df2bac741a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:29.981301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:29.983925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:30.003983 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-87423bcd-49b5-4b21-9b49-df2bac741a7e None None] MFA Rules not processed for user `e574095b22114893b590b94d83439574`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:30.005485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-87423bcd-49b5-4b21-9b49-df2bac741a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:30.023123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:30.069983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d941e5-6024-4f4a-8ae3-915ca696d629 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:30.074300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:30.075683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.075960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.076250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.078109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:30.094622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1276/2547] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:03:30.100061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87423bcd-49b5-4b21-9b49-df2bac741a7e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:30.154353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:30.155784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:30.168229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1272/2548] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:03:30.175959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87423bcd-49b5-4b21-9b49-df2bac741a7e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:30.184713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc7ec-92ba-4564-9ed3-ba51249ceee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:30.185333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc7ec-92ba-4564-9ed3-ba51249ceee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.185614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc7ec-92ba-4564-9ed3-ba51249ceee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.185885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fc7ec-92ba-4564-9ed3-ba51249ceee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.186514 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fc7ec-92ba-4564-9ed3-ba51249ceee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:30.188468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fc7ec-92ba-4564-9ed3-ba51249ceee6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:30.227349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1277/2549] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:30 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:30.228386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:30.238784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1c5b6-1431-41ef-b6bf-3b29263e4db5 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:30.304935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91b7fb-8b82-4f94-803a-585c05e78279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:30.305544 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91b7fb-8b82-4f94-803a-585c05e78279 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.306293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91b7fb-8b82-4f94-803a-585c05e78279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.306786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91b7fb-8b82-4f94-803a-585c05e78279 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.307631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91b7fb-8b82-4f94-803a-585c05e78279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:30.309793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91b7fb-8b82-4f94-803a-585c05e78279 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:30.336171 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1273/2550] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:30 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:30.339343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc7ec-92ba-4564-9ed3-ba51249ceee6 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:30.351587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fc7ec-92ba-4564-9ed3-ba51249ceee6 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:30.357035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651bb2-3716-4cfd-8717-8f3d201cc6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:30.357641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651bb2-3716-4cfd-8717-8f3d201cc6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.357917 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651bb2-3716-4cfd-8717-8f3d201cc6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.358193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55651bb2-3716-4cfd-8717-8f3d201cc6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.358985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651bb2-3716-4cfd-8717-8f3d201cc6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:30.360973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55651bb2-3716-4cfd-8717-8f3d201cc6ce admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:30.375127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1278/2551] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:30.379852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91b7fb-8b82-4f94-803a-585c05e78279 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:30.390964 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91b7fb-8b82-4f94-803a-585c05e78279 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:30.397104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee53118-7a59-4905-ab69-6a9547598056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:30.398159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee53118-7a59-4905-ab69-6a9547598056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.398683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee53118-7a59-4905-ab69-6a9547598056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.399135 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee53118-7a59-4905-ab69-6a9547598056 admin admin] PATH_INFO: `/v3/projects/7f8306262a674a208f8a780eaad87a3e/users/eb63959015984b9792e74ff974cc3d97/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.416235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee53118-7a59-4905-ab69-6a9547598056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8306262a674a208f8a780eaad87a3e, user_id=eb63959015984b9792e74ff974cc3d97, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:30.419104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee53118-7a59-4905-ab69-6a9547598056 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:30.436033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1274/2552] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:30 2022] PUT /identity/v3/projects/7f8306262a674a208f8a780eaad87a3e/users/eb63959015984b9792e74ff974cc3d97/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:30.440004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651bb2-3716-4cfd-8717-8f3d201cc6ce admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:30.454095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55651bb2-3716-4cfd-8717-8f3d201cc6ce admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:30.459912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:30.529736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f76fcb-cba6-49a6-aae4-0a8a526dd9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:30.530347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f76fcb-cba6-49a6-aae4-0a8a526dd9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.530626 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f76fcb-cba6-49a6-aae4-0a8a526dd9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.530892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f76fcb-cba6-49a6-aae4-0a8a526dd9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.531542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f76fcb-cba6-49a6-aae4-0a8a526dd9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:30.533155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f76fcb-cba6-49a6-aae4-0a8a526dd9b3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:30.544411 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1279/2553] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:30.548439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee53118-7a59-4905-ab69-6a9547598056 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:30.559818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee53118-7a59-4905-ab69-6a9547598056 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:30.565954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:30.566852 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.567315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.567767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] PATH_INFO: `/v3/projects/7f8306262a674a208f8a780eaad87a3e/users/eb63959015984b9792e74ff974cc3d97/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.569801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8306262a674a208f8a780eaad87a3e, user_id=eb63959015984b9792e74ff974cc3d97, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:30.572827 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:30.594532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1275/2554] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:30 2022] PUT /identity/v3/projects/7f8306262a674a208f8a780eaad87a3e/users/eb63959015984b9792e74ff974cc3d97/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:30.610407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa201-5ee6-4e03-a410-7ac7ca209643 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.610407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa201-5ee6-4e03-a410-7ac7ca209643 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.610407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa201-5ee6-4e03-a410-7ac7ca209643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.640807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-96cfa201-5ee6-4e03-a410-7ac7ca209643 None None] MFA Rules not processed for user `eb63959015984b9792e74ff974cc3d97`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:30.642161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-96cfa201-5ee6-4e03-a410-7ac7ca209643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:30.748767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1280/2555] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:30.763841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:30.765625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:30.820314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:30.849702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25056658-9f53-4283-b8a9-50ac4e89a88a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:30.855949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85086616-39c0-47fd-ac4e-90a9a6b1acb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:30.856946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85086616-39c0-47fd-ac4e-90a9a6b1acb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:30.857702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85086616-39c0-47fd-ac4e-90a9a6b1acb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:30.858237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85086616-39c0-47fd-ac4e-90a9a6b1acb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:30.871777 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85086616-39c0-47fd-ac4e-90a9a6b1acb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:30.873663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85086616-39c0-47fd-ac4e-90a9a6b1acb0 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:30.892123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1276/2556] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:31.379439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfa201-5ee6-4e03-a410-7ac7ca209643 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:31.450462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfa201-5ee6-4e03-a410-7ac7ca209643 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:31.456020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:31.456877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:31.457340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:31.457850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] PATH_INFO: `/v3/users/e574095b22114893b590b94d83439574` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:31.459172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e574095b22114893b590b94d83439574)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:31.461317 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:31.507638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:31.527361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e574095b22114893b590b94d83439574'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:31.541288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e574095b22114893b590b94d83439574'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:31.542071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e574095b22114893b590b94d83439574'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:31.550357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e574095b22114893b590b94d83439574'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:31.561614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e574095b22114893b590b94d83439574'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:31.574687 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1281/2557] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:31 2022] DELETE /identity/v3/users/e574095b22114893b590b94d83439574 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:31.577203 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe156-3dcd-42b4-83df-8a83fea29bb7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:31.578069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe156-3dcd-42b4-83df-8a83fea29bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:31.578364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe156-3dcd-42b4-83df-8a83fea29bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:31.600525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-984fe156-3dcd-42b4-83df-8a83fea29bb7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:31.601780 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-984fe156-3dcd-42b4-83df-8a83fea29bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:31.672421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1277/2558] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:31.683604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:31.685131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:31.769970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff06799-ec47-407a-becb-9f3e217adf49 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:31.774155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c215a-8964-4141-986a-54873b02c4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:31.774851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c215a-8964-4141-986a-54873b02c4bd service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:31.775137 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c215a-8964-4141-986a-54873b02c4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:31.775410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c215a-8964-4141-986a-54873b02c4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:31.795663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c215a-8964-4141-986a-54873b02c4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:31.797299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c215a-8964-4141-986a-54873b02c4bd service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:31.814023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1282/2559] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:03:32.078861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fe156-3dcd-42b4-83df-8a83fea29bb7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:32.080512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-984fe156-3dcd-42b4-83df-8a83fea29bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:32.098026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fe156-3dcd-42b4-83df-8a83fea29bb7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:32.116756 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:32.117467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:32.117827 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:32.118673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] PATH_INFO: `/v3/users/f87cc3e4f0954c0388d277913bfe3285` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:32.120606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87cc3e4f0954c0388d277913bfe3285)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:32.123074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:32.214035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87cc3e4f0954c0388d277913bfe3285'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.225761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87cc3e4f0954c0388d277913bfe3285'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.226546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87cc3e4f0954c0388d277913bfe3285'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.248949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87cc3e4f0954c0388d277913bfe3285'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.249760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87cc3e4f0954c0388d277913bfe3285'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.275083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1278/2560] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:32 2022] DELETE /identity/v3/users/f87cc3e4f0954c0388d277913bfe3285 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:32.570148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c215a-8964-4141-986a-54873b02c4bd service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:32.580751 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c215a-8964-4141-986a-54873b02c4bd service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:32.594722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:32.599088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:32.655828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:32.656660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:32.657043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:32.657611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] PATH_INFO: `/v3/projects/5fd0a27a2f13490eb9d5a5b74956df31` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:32.659362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd0a27a2f13490eb9d5a5b74956df31)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:32.661832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:32.670047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:32.690990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07b49e-9887-4d20-8d56-6557d74ef079 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:32.701433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:32.702167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:32.702445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:32.702720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] PATH_INFO: `/v3/projects/47e4979b4eee43989590f7e2fbc50204` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:32.703991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] RBAC: Authorizing `identity:delete_project(project_id=47e4979b4eee43989590f7e2fbc50204)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:32.707224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:32.725412 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd0a27a2f13490eb9d5a5b74956df31', 'request_id': 'req-7e9f3613-0995-4935-bd29-3c28da7fab9e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.728358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:32.734590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd0a27a2f13490eb9d5a5b74956df31', 'request_id': 'req-7e9f3613-0995-4935-bd29-3c28da7fab9e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.744132 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd0a27a2f13490eb9d5a5b74956df31', 'request_id': 'req-7e9f3613-0995-4935-bd29-3c28da7fab9e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.758331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] The token cache is being invalidate because project 5fd0a27a2f13490eb9d5a5b74956df31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:32.758877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.759599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.761487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1283/2561] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:32 2022] DELETE /identity/v3/projects/5fd0a27a2f13490eb9d5a5b74956df31 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:32.761649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e4979b4eee43989590f7e2fbc50204', 'request_id': 'req-0634278a-7b9b-4da8-91e7-8ab853cf772f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.765555 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:32.766999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:32.771155 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e4979b4eee43989590f7e2fbc50204', 'request_id': 'req-0634278a-7b9b-4da8-91e7-8ab853cf772f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.781000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e4979b4eee43989590f7e2fbc50204', 'request_id': 'req-0634278a-7b9b-4da8-91e7-8ab853cf772f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.797520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] The token cache is being invalidate because project 47e4979b4eee43989590f7e2fbc50204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:32.797790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.798504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:32.800235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1279/2562] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:32 2022] DELETE /identity/v3/projects/47e4979b4eee43989590f7e2fbc50204 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:32.804482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:32.806083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:32.866517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f3613-0995-4935-bd29-3c28da7fab9e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:32.902392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634278a-7b9b-4da8-91e7-8ab853cf772f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:32.908074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:32.920765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:32.922194 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:32.924147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] PATH_INFO: `/v3/users/eb63959015984b9792e74ff974cc3d97` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:32.927346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb63959015984b9792e74ff974cc3d97)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:32.932978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:32.937749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:32.938553 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:32.938979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:32.940911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] PATH_INFO: `/v3/users/d250554ebee44a679240cdf4eaf6542e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:32.942328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d250554ebee44a679240cdf4eaf6542e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:32.944550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:33.029744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb63959015984b9792e74ff974cc3d97'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.039357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd250554ebee44a679240cdf4eaf6542e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.042810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb63959015984b9792e74ff974cc3d97'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.047582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb63959015984b9792e74ff974cc3d97'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.065757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd250554ebee44a679240cdf4eaf6542e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.073775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd250554ebee44a679240cdf4eaf6542e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.077371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb63959015984b9792e74ff974cc3d97'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.078344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb63959015984b9792e74ff974cc3d97'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.092225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:33.095600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd250554ebee44a679240cdf4eaf6542e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.110010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd250554ebee44a679240cdf4eaf6542e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.117315 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1280/2563] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:32 2022] DELETE /identity/v3/users/eb63959015984b9792e74ff974cc3d97 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:03:33.124672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1284/2564] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:32 2022] DELETE /identity/v3/users/d250554ebee44a679240cdf4eaf6542e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:33.550360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:33.551833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:33.649447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:33.651210 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:33.664525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:33.685226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa741ee9-5d79-42c9-846e-7d9c7f0e60c5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:33.771165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:33.771878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:33.772595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:33.772595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] PATH_INFO: `/v3/projects/06f5a1e96f2b4ffcbf518de535440e33` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:33.773718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f5a1e96f2b4ffcbf518de535440e33)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:33.776437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:33.784233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e4b69-9a49-4ec8-b24b-a7e55cf213c2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:33.797549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:33.798761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:33.799533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:33.800391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] PATH_INFO: `/v3/projects/7f8306262a674a208f8a780eaad87a3e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:33.802536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f8306262a674a208f8a780eaad87a3e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:33.806357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:33.847765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f5a1e96f2b4ffcbf518de535440e33', 'request_id': 'req-e47b1087-ec1d-4339-8584-444f60b9d322'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.860554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f5a1e96f2b4ffcbf518de535440e33', 'request_id': 'req-e47b1087-ec1d-4339-8584-444f60b9d322'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.875566 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f5a1e96f2b4ffcbf518de535440e33', 'request_id': 'req-e47b1087-ec1d-4339-8584-444f60b9d322'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.892475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] The token cache is being invalidate because project 06f5a1e96f2b4ffcbf518de535440e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:33.893227 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.894150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e47b1087-ec1d-4339-8584-444f60b9d322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.899539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1281/2565] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:33 2022] DELETE /identity/v3/projects/06f5a1e96f2b4ffcbf518de535440e33 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:33.900907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785058c4-db9d-47c0-82a0-fe3653ce67fa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:33.901198 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785058c4-db9d-47c0-82a0-fe3653ce67fa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:33.901480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785058c4-db9d-47c0-82a0-fe3653ce67fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:33.927302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f8306262a674a208f8a780eaad87a3e', 'request_id': 'req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.931068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-785058c4-db9d-47c0-82a0-fe3653ce67fa None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:33.932797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-785058c4-db9d-47c0-82a0-fe3653ce67fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:33.939152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f8306262a674a208f8a780eaad87a3e', 'request_id': 'req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.950811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f8306262a674a208f8a780eaad87a3e', 'request_id': 'req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.984391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:33.996015 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] The token cache is being invalidate because project 7f8306262a674a208f8a780eaad87a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:33.996445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:33.997329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ea2fedc-51c1-4f04-8e15-5d8f7bc202f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.004694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1285/2566] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:33 2022] DELETE /identity/v3/projects/7f8306262a674a208f8a780eaad87a3e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:34.008129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0dd5c-f980-4503-b144-95b61c1d4227 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.008470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0dd5c-f980-4503-b144-95b61c1d4227 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.008717 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f0dd5c-f980-4503-b144-95b61c1d4227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.024968 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1282/2567] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:03:34.032843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774f800-bb47-4984-8615-77b03e56f750 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.033229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774f800-bb47-4984-8615-77b03e56f750 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.033627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774f800-bb47-4984-8615-77b03e56f750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.038518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d9f0dd5c-f980-4503-b144-95b61c1d4227 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:34.039940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d9f0dd5c-f980-4503-b144-95b61c1d4227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.053764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1286/2568] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:03:34.055767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0dd5c-f980-4503-b144-95b61c1d4227 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.062065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d9f0dd5c-f980-4503-b144-95b61c1d4227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.069610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-b774f800-bb47-4984-8615-77b03e56f750 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:34.070937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b774f800-bb47-4984-8615-77b03e56f750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.083312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1283/2569] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:34.083312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a749-997a-4c56-a460-1d60dac01153 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.083312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a749-997a-4c56-a460-1d60dac01153 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.083312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a749-997a-4c56-a460-1d60dac01153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.108736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c4b0a749-997a-4c56-a460-1d60dac01153 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:34.110110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c4b0a749-997a-4c56-a460-1d60dac01153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.124187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1284/2570] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:34.124187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0a749-997a-4c56-a460-1d60dac01153 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.132797 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0a749-997a-4c56-a460-1d60dac01153 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.137499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.138110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.138522 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.138635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.139284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:03:34.139605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.141201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.156188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1285/2571] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:03:34.161353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.166837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.168480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f0dd5c-f980-4503-b144-95b61c1d4227 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.178802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.179673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.180294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.180294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.182019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.185036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf4aa8-7b85-4a18-89ee-6bbcbcaa103b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.190285 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.191423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.191773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.192249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] PATH_INFO: `/v3/users/5d59680b3fde47bf81c7765ca4d8a757` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.196165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.198206 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.209061 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d59680b3fde47bf81c7765ca4d8a757)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.212014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.215413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1287/2572] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:34.218061 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.232775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e9e0b-3569-45e3-bfc5-e2666358ebd9 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.243000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.243715 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.243964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.244253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.244965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:03:34.245279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.247614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.269123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1288/2573] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:03:34.273732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.285842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129feca0-aba7-4e95-9bfd-2fce6edff8f1 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.293606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66016217-0889-45e9-87ba-f4dc234d90fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.294388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016217-0889-45e9-87ba-f4dc234d90fe admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.294713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016217-0889-45e9-87ba-f4dc234d90fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.295077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66016217-0889-45e9-87ba-f4dc234d90fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.295804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66016217-0889-45e9-87ba-f4dc234d90fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.298248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66016217-0889-45e9-87ba-f4dc234d90fe admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.304026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:34.322731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d59680b3fde47bf81c7765ca4d8a757'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.333723 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d59680b3fde47bf81c7765ca4d8a757'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.334739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d59680b3fde47bf81c7765ca4d8a757'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.339649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1289/2574] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:34.340496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66016217-0889-45e9-87ba-f4dc234d90fe admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.350528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d59680b3fde47bf81c7765ca4d8a757'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.351219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d59680b3fde47bf81c7765ca4d8a757'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.364080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1286/2575] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:34 2022] DELETE /identity/v3/users/5d59680b3fde47bf81c7765ca4d8a757 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:34.364938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.366279 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.407416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66016217-0889-45e9-87ba-f4dc234d90fe admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.444435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c1004-d25b-40e7-9905-5c2acb810fa1 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.449783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.450663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.450940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.451609 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.451819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.453775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.476215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:34.489131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1287/2576] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:34.490538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.493746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.507410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ee0d9-8e35-41f5-8ec6-2d00465ac990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.508047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ee0d9-8e35-41f5-8ec6-2d00465ac990 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.508254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ee0d9-8e35-41f5-8ec6-2d00465ac990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.508796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ee0d9-8e35-41f5-8ec6-2d00465ac990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.509208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ee0d9-8e35-41f5-8ec6-2d00465ac990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.511112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ee0d9-8e35-41f5-8ec6-2d00465ac990 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.542181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1290/2577] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:34.546398 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ee0d9-8e35-41f5-8ec6-2d00465ac990 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.572947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154deede-dc35-446b-83b2-c65f227e15ef admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.617104 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ee0d9-8e35-41f5-8ec6-2d00465ac990 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.622234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.622840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.623145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.623424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.624077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.625698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.631445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.632339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.632619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.632890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.634655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.639071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1291/2578] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:34.639071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.639071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.647857 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.649443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.651443 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ed3a8-c16a-4994-b905-b0a4da3c5592 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.656319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ec7bc-79a0-4633-9654-bf3f9d28dabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.656937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ec7bc-79a0-4633-9654-bf3f9d28dabc admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.657220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ec7bc-79a0-4633-9654-bf3f9d28dabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.657547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ec7bc-79a0-4633-9654-bf3f9d28dabc admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.658169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ec7bc-79a0-4633-9654-bf3f9d28dabc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.660388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1288/2579] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:34.661058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ec7bc-79a0-4633-9654-bf3f9d28dabc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.661204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.662494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.672141 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:34.675391 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345fee7-a1ec-4e3b-ba56-2e45f2463277 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.681940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.682671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.683022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.683843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] PATH_INFO: `/v3/projects/0ea665a47e44438aa8c4a9e763c78fdd` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.683994 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1292/2580] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:34.684914 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea665a47e44438aa8c4a9e763c78fdd)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.686915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.687511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ec7bc-79a0-4633-9654-bf3f9d28dabc admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.697797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ec7bc-79a0-4633-9654-bf3f9d28dabc admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.703274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde4239-b639-4a2b-a679-6e536358b6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.703892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4239-b639-4a2b-a679-6e536358b6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.704176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4239-b639-4a2b-a679-6e536358b6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.704500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde4239-b639-4a2b-a679-6e536358b6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.705921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde4239-b639-4a2b-a679-6e536358b6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.707551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde4239-b639-4a2b-a679-6e536358b6e2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.721695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1293/2581] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:34.722973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde4239-b639-4a2b-a679-6e536358b6e2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.733337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde4239-b639-4a2b-a679-6e536358b6e2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.745849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea665a47e44438aa8c4a9e763c78fdd', 'request_id': 'req-f87b7777-442f-442a-a1be-4ef6e85db207'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.754300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea665a47e44438aa8c4a9e763c78fdd', 'request_id': 'req-f87b7777-442f-442a-a1be-4ef6e85db207'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.761787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea665a47e44438aa8c4a9e763c78fdd', 'request_id': 'req-f87b7777-442f-442a-a1be-4ef6e85db207'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.775879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] The token cache is being invalidate because project 0ea665a47e44438aa8c4a9e763c78fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:34.776401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.777073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:34.781876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1289/2582] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:34 2022] DELETE /identity/v3/projects/0ea665a47e44438aa8c4a9e763c78fdd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:34.781876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.781876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.815382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b9ecf-8cb5-4c40-8e1a-010daf9b10d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.815995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b9ecf-8cb5-4c40-8e1a-010daf9b10d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.816377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b9ecf-8cb5-4c40-8e1a-010daf9b10d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.816648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b9ecf-8cb5-4c40-8e1a-010daf9b10d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.818177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b9ecf-8cb5-4c40-8e1a-010daf9b10d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.818869 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b9ecf-8cb5-4c40-8e1a-010daf9b10d0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.833903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1294/2583] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:34.833903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaacc2-4860-4b03-a5ea-199c43692d4d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.833903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaacc2-4860-4b03-a5ea-199c43692d4d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.833903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaacc2-4860-4b03-a5ea-199c43692d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.836053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:34.855807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-bedaacc2-4860-4b03-a5ea-199c43692d4d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:34.857606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-bedaacc2-4860-4b03-a5ea-199c43692d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.859549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b7777-442f-442a-a1be-4ef6e85db207 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.865564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cd2d4-c937-4193-af51-0fe5e09df99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.866286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cd2d4-c937-4193-af51-0fe5e09df99f admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.866633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cd2d4-c937-4193-af51-0fe5e09df99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.867262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cd2d4-c937-4193-af51-0fe5e09df99f admin admin] PATH_INFO: `/v3/projects/6b5a2925403842509420d0693ecb9570/users/7cba2c8197d340cc8fb9a47fbb7a5460/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.868827 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1295/2584] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:34.873017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaacc2-4860-4b03-a5ea-199c43692d4d None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.874837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-bedaacc2-4860-4b03-a5ea-199c43692d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:34.888765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaacc2-4860-4b03-a5ea-199c43692d4d None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:34.891540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cd2d4-c937-4193-af51-0fe5e09df99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b5a2925403842509420d0693ecb9570, user_id=7cba2c8197d340cc8fb9a47fbb7a5460, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.894966 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cd2d4-c937-4193-af51-0fe5e09df99f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.896665 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cbe9e-b79f-40c4-90e7-7339907bd19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:34.898963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cbe9e-b79f-40c4-90e7-7339907bd19a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.899947 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cbe9e-b79f-40c4-90e7-7339907bd19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.900172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cbe9e-b79f-40c4-90e7-7339907bd19a admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.900956 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cbe9e-b79f-40c4-90e7-7339907bd19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:34.902733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cbe9e-b79f-40c4-90e7-7339907bd19a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:34.919073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1296/2586] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:34.919073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af6b2f-f722-427a-a60a-107575fe3dca None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:34.919073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af6b2f-f722-427a-a60a-107575fe3dca None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:34.919073 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af6b2f-f722-427a-a60a-107575fe3dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:34.922680 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1290/2586] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:34 2022] PUT /identity/v3/projects/6b5a2925403842509420d0693ecb9570/users/7cba2c8197d340cc8fb9a47fbb7a5460/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:34.923627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cd2d4-c937-4193-af51-0fe5e09df99f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:34.938691 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-91af6b2f-f722-427a-a60a-107575fe3dca None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:34.940164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-91af6b2f-f722-427a-a60a-107575fe3dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.004178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cd2d4-c937-4193-af51-0fe5e09df99f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.005159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1297/2587] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:35.006455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899bc2b-1a0e-4707-837a-07efbc2bb20b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.006748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899bc2b-1a0e-4707-837a-07efbc2bb20b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.007020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899bc2b-1a0e-4707-837a-07efbc2bb20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.012832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033a88f-596d-4597-90c6-f0a6a1105b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.013515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033a88f-596d-4597-90c6-f0a6a1105b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.013799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033a88f-596d-4597-90c6-f0a6a1105b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.014362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033a88f-596d-4597-90c6-f0a6a1105b15 admin admin] PATH_INFO: `/v3/projects/7e001a3e5ab54b6180868a9a557a035e/users/a248a53e6b6b43fe9f4cf9547a939bc7/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.034354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033a88f-596d-4597-90c6-f0a6a1105b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e001a3e5ab54b6180868a9a557a035e, user_id=a248a53e6b6b43fe9f4cf9547a939bc7, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.036851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1899bc2b-1a0e-4707-837a-07efbc2bb20b None None] MFA Rules not processed for user `7cba2c8197d340cc8fb9a47fbb7a5460`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:35.037449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033a88f-596d-4597-90c6-f0a6a1105b15 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.038087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1899bc2b-1a0e-4707-837a-07efbc2bb20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.059319 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1291/2588] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:34 2022] PUT /identity/v3/projects/7e001a3e5ab54b6180868a9a557a035e/users/a248a53e6b6b43fe9f4cf9547a939bc7/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:35.064506 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033a88f-596d-4597-90c6-f0a6a1105b15 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.118077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:35.132958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1298/2589] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:35.134531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3101-ae61-433e-9880-61c6808ab730 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.134811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3101-ae61-433e-9880-61c6808ab730 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.135074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd3101-ae61-433e-9880-61c6808ab730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.139664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033a88f-596d-4597-90c6-f0a6a1105b15 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.145313 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.145915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.146322 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.146543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.147259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:03:35.147504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.149475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.165582 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1292/2590] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:03:35.167197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.170306 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d0bd3101-ae61-433e-9880-61c6808ab730 None None] MFA Rules not processed for user `a248a53e6b6b43fe9f4cf9547a939bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:35.170557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.171648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d0bd3101-ae61-433e-9880-61c6808ab730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.254664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:35.254919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1299/2591] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:35.255773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd3101-ae61-433e-9880-61c6808ab730 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.269391 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd3101-ae61-433e-9880-61c6808ab730 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.275262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ba113-197b-4533-86c3-399ef8067079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.275941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ba113-197b-4533-86c3-399ef8067079 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.276272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ba113-197b-4533-86c3-399ef8067079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.276457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ba113-197b-4533-86c3-399ef8067079 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.277095 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ba113-197b-4533-86c3-399ef8067079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.280202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ba113-197b-4533-86c3-399ef8067079 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.296067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8b84-badc-4e64-9b97-0cf5768d3289 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.301008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7a2237c18934cb2aa3699a5776874ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.301685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.302104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.302469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.310608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1300/2592] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:35 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:35.311748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ba113-197b-4533-86c3-399ef8067079 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.371739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.373249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.380852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ba113-197b-4533-86c3-399ef8067079 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.382759 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1293/2593] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:35.383576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.385054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.386611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723214f-2dd8-4837-b8be-3046620a7571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.387328 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723214f-2dd8-4837-b8be-3046620a7571 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.387669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723214f-2dd8-4837-b8be-3046620a7571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.388113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723214f-2dd8-4837-b8be-3046620a7571 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.388899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723214f-2dd8-4837-b8be-3046620a7571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.391058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723214f-2dd8-4837-b8be-3046620a7571 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.402608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a672f-9b9e-4202-a875-2704e8345435 service glance] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.412217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.412870 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.413153 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.414079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] PATH_INFO: `/v3/users/992a34be52b7491da0b68cbe17f25b39` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.415420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=992a34be52b7491da0b68cbe17f25b39)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.417377 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.435224 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1301/2594] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:35 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:35.436487 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723214f-2dd8-4837-b8be-3046620a7571 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.492131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992a34be52b7491da0b68cbe17f25b39'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.510631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992a34be52b7491da0b68cbe17f25b39'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.511702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992a34be52b7491da0b68cbe17f25b39'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.516905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:35.529583 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992a34be52b7491da0b68cbe17f25b39'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.530421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992a34be52b7491da0b68cbe17f25b39'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.536842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723214f-2dd8-4837-b8be-3046620a7571 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.542519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.543461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.543940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.544401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.545208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.547277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.548177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1294/2595] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:35 2022] DELETE /identity/v3/users/992a34be52b7491da0b68cbe17f25b39 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:35.553160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.555158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.578130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1302/2596] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:35 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:35.581920 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.589581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.607087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c99238-06aa-4868-8296-442e1ca2312f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.645097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:35.666106 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946a5ab-c3fa-43a0-9049-fd6b6e325ffc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.672076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d263be-c386-4576-8796-26dd8c9165bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.672747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d263be-c386-4576-8796-26dd8c9165bf admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.673246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d263be-c386-4576-8796-26dd8c9165bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.673671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d263be-c386-4576-8796-26dd8c9165bf admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.674901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d263be-c386-4576-8796-26dd8c9165bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.676849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d263be-c386-4576-8796-26dd8c9165bf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.681650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbcee0-32e1-40c0-986c-f40c22856a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.682266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbcee0-32e1-40c0-986c-f40c22856a83 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.682548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbcee0-32e1-40c0-986c-f40c22856a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.682816 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbcee0-32e1-40c0-986c-f40c22856a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.683469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbcee0-32e1-40c0-986c-f40c22856a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.685143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbcee0-32e1-40c0-986c-f40c22856a83 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.701529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1303/2597] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:03:35.702776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbcee0-32e1-40c0-986c-f40c22856a83 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.710225 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1295/2598] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:35 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:35.714542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbcee0-32e1-40c0-986c-f40c22856a83 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.717400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d263be-c386-4576-8796-26dd8c9165bf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.724191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb99ef-49e2-4383-93b5-19f58949767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.725047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb99ef-49e2-4383-93b5-19f58949767e admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.725518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb99ef-49e2-4383-93b5-19f58949767e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.726152 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb99ef-49e2-4383-93b5-19f58949767e admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.727037 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb99ef-49e2-4383-93b5-19f58949767e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.728811 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb99ef-49e2-4383-93b5-19f58949767e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.740075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d263be-c386-4576-8796-26dd8c9165bf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.744647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1304/2599] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:35.745502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb99ef-49e2-4383-93b5-19f58949767e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.752479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.753480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.753781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.754060 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.754719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.756584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.764738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb99ef-49e2-4383-93b5-19f58949767e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.770278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1296/2600] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:35.770988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.773001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.777143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558af5d0-55fb-4ffb-a64b-44bafeac9a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.778181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af5d0-55fb-4ffb-a64b-44bafeac9a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.778667 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af5d0-55fb-4ffb-a64b-44bafeac9a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.779282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558af5d0-55fb-4ffb-a64b-44bafeac9a0d admin admin] PATH_INFO: `/v3/projects/6432e6a1173b4fec9083fb94936b069e/users/e0accb8344df4cd1b3959212b654f66b/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.792975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dc064-9229-4ab2-82f4-c3348ad03838 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.798486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558af5d0-55fb-4ffb-a64b-44bafeac9a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6432e6a1173b4fec9083fb94936b069e, user_id=e0accb8344df4cd1b3959212b654f66b, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.798834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:35.799474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.799753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.800025 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] PATH_INFO: `/v3/projects/92e71ba59c6d42dabeef49f62cebac1d` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.801217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e71ba59c6d42dabeef49f62cebac1d)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:35.803119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558af5d0-55fb-4ffb-a64b-44bafeac9a0d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.803291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:35.821049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1305/2601] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:35 2022] PUT /identity/v3/projects/6432e6a1173b4fec9083fb94936b069e/users/e0accb8344df4cd1b3959212b654f66b/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:35.821944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558af5d0-55fb-4ffb-a64b-44bafeac9a0d admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:35.868855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e71ba59c6d42dabeef49f62cebac1d', 'request_id': 'req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.877634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e71ba59c6d42dabeef49f62cebac1d', 'request_id': 'req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.888261 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e71ba59c6d42dabeef49f62cebac1d', 'request_id': 'req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.903192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558af5d0-55fb-4ffb-a64b-44bafeac9a0d admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:35.903661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] The token cache is being invalidate because project 92e71ba59c6d42dabeef49f62cebac1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:35.903937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.904690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-bc97e0aa-9f06-4bd7-bc86-f56dc338b0a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:35.906501 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1297/2602] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:35 2022] DELETE /identity/v3/projects/92e71ba59c6d42dabeef49f62cebac1d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:35.910267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e6725-c329-47f3-9a72-2808494b03a6 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:35.910601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e6725-c329-47f3-9a72-2808494b03a6 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:35.911011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e6725-c329-47f3-9a72-2808494b03a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:35.951065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-862e6725-c329-47f3-9a72-2808494b03a6 None None] MFA Rules not processed for user `e0accb8344df4cd1b3959212b654f66b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:35.951065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-862e6725-c329-47f3-9a72-2808494b03a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:35.980051 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:35.999960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.000844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.001191 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.001354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] PATH_INFO: `/v3/projects/014a1b871a6b4546aa4bd05b66aba241/users/6fe0c17a6d7342deb62e7bc86715478f/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.019729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014a1b871a6b4546aa4bd05b66aba241, user_id=6fe0c17a6d7342deb62e7bc86715478f, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.023279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.026832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1298/2603] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:36.028357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1eab2-d656-48e0-8409-af7e72ef2738 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.028701 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1eab2-d656-48e0-8409-af7e72ef2738 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.029076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1eab2-d656-48e0-8409-af7e72ef2738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.042512 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1306/2604] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:35 2022] PUT /identity/v3/projects/014a1b871a6b4546aa4bd05b66aba241/users/6fe0c17a6d7342deb62e7bc86715478f/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:36.044172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.045968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.051616 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-03f1eab2-d656-48e0-8409-af7e72ef2738 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:36.053185 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-03f1eab2-d656-48e0-8409-af7e72ef2738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.091657 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:36.119727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1299/2605] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:36.120697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1eab2-d656-48e0-8409-af7e72ef2738 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.122003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-03f1eab2-d656-48e0-8409-af7e72ef2738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.128766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb01bd-b27b-4a82-af1a-598ae5e683b5 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.134336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c7f8f-4ffe-4bf7-94bd-197e226c7c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.134515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1eab2-d656-48e0-8409-af7e72ef2738 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.134948 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c7f8f-4ffe-4bf7-94bd-197e226c7c88 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.135227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c7f8f-4ffe-4bf7-94bd-197e226c7c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.135499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c7f8f-4ffe-4bf7-94bd-197e226c7c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.136164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c7f8f-4ffe-4bf7-94bd-197e226c7c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.138193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c7f8f-4ffe-4bf7-94bd-197e226c7c88 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.139262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c1bdf-d78f-4f69-b34a-a56d23cfafae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.139895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c1bdf-d78f-4f69-b34a-a56d23cfafae admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.140204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c1bdf-d78f-4f69-b34a-a56d23cfafae admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.140478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c1bdf-d78f-4f69-b34a-a56d23cfafae admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.141121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c1bdf-d78f-4f69-b34a-a56d23cfafae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.142781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c1bdf-d78f-4f69-b34a-a56d23cfafae admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.153662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1300/2606] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:36.154905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a77183-7896-461b-a083-a1f29322a7f1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.155170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a77183-7896-461b-a083-a1f29322a7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.155485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a77183-7896-461b-a083-a1f29322a7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.169708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1307/2607] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/projects => generated 361 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:36.170863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c7f8f-4ffe-4bf7-94bd-197e226c7c88 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.182091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-74a77183-7896-461b-a083-a1f29322a7f1 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:36.183346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-74a77183-7896-461b-a083-a1f29322a7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.239517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c7f8f-4ffe-4bf7-94bd-197e226c7c88 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.244742 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb541942-5d9c-4d42-a739-35972743cac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.245401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb541942-5d9c-4d42-a739-35972743cac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.245682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb541942-5d9c-4d42-a739-35972743cac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.246078 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb541942-5d9c-4d42-a739-35972743cac8 admin admin] PATH_INFO: `/v3/projects/014a1b871a6b4546aa4bd05b66aba241/users/6fe0c17a6d7342deb62e7bc86715478f/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.247914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb541942-5d9c-4d42-a739-35972743cac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014a1b871a6b4546aa4bd05b66aba241, user_id=6fe0c17a6d7342deb62e7bc86715478f, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.250647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb541942-5d9c-4d42-a739-35972743cac8 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.255635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1301/2608] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:36.256704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a77183-7896-461b-a083-a1f29322a7f1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.261835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1308/2609] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:36 2022] PUT /identity/v3/projects/014a1b871a6b4546aa4bd05b66aba241/users/6fe0c17a6d7342deb62e7bc86715478f/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:36.262496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb541942-5d9c-4d42-a739-35972743cac8 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.266946 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a77183-7896-461b-a083-a1f29322a7f1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.321335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:36.341822 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb541942-5d9c-4d42-a739-35972743cac8 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.345477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2096bd-61f6-44b3-8fe7-6c5719954424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.346755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2096bd-61f6-44b3-8fe7-6c5719954424 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.347211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2096bd-61f6-44b3-8fe7-6c5719954424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.347399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2096bd-61f6-44b3-8fe7-6c5719954424 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.348018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2096bd-61f6-44b3-8fe7-6c5719954424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.348209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62b8ae-e6ca-4a63-a803-32167f504af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.348750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62b8ae-e6ca-4a63-a803-32167f504af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.349033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62b8ae-e6ca-4a63-a803-32167f504af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.349330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62b8ae-e6ca-4a63-a803-32167f504af5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.349954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62b8ae-e6ca-4a63-a803-32167f504af5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:03:36.350178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2096bd-61f6-44b3-8fe7-6c5719954424 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.350359 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62b8ae-e6ca-4a63-a803-32167f504af5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.351838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62b8ae-e6ca-4a63-a803-32167f504af5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.365234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1309/2610] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:03:36.366745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f39c1b-c5a1-4fc6-992e-6c25afd8e96a None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.367048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f39c1b-c5a1-4fc6-992e-6c25afd8e96a None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.367364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f39c1b-c5a1-4fc6-992e-6c25afd8e96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.377371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1302/2611] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:36.378529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2096bd-61f6-44b3-8fe7-6c5719954424 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.388781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2096bd-61f6-44b3-8fe7-6c5719954424 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.394912 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7e5eb-3841-4657-bf51-7c2865ce982b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.395523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e5eb-3841-4657-bf51-7c2865ce982b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.395804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e5eb-3841-4657-bf51-7c2865ce982b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.396087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e5eb-3841-4657-bf51-7c2865ce982b admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.396985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7e5eb-3841-4657-bf51-7c2865ce982b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.398917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7e5eb-3841-4657-bf51-7c2865ce982b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.400842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-12f39c1b-c5a1-4fc6-992e-6c25afd8e96a None None] MFA Rules not processed for user `6fe0c17a6d7342deb62e7bc86715478f`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:36.402127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-12f39c1b-c5a1-4fc6-992e-6c25afd8e96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.431086 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1303/2612] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:36.431848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7e5eb-3841-4657-bf51-7c2865ce982b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.481617 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:36.492602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1310/2613] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:36.494897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f39c1b-c5a1-4fc6-992e-6c25afd8e96a None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.509792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f39c1b-c5a1-4fc6-992e-6c25afd8e96a None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.511526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7e5eb-3841-4657-bf51-7c2865ce982b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.515633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1897c-8ebe-4441-b7ad-70250bdc02e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.516276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1897c-8ebe-4441-b7ad-70250bdc02e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.516558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1897c-8ebe-4441-b7ad-70250bdc02e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.516833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1897c-8ebe-4441-b7ad-70250bdc02e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.517466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1897c-8ebe-4441-b7ad-70250bdc02e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.519143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.519419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1897c-8ebe-4441-b7ad-70250bdc02e3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.519748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.520899 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.521180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.521971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.523606 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.544201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1304/2614] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:36.545238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.547211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.556172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1311/2615] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/users => generated 292 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:36.556172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1897c-8ebe-4441-b7ad-70250bdc02e3 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.567779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1897c-8ebe-4441-b7ad-70250bdc02e3 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.572845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e8a44-472e-4908-b28d-7610cc3fba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.573509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8a44-472e-4908-b28d-7610cc3fba78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.573793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8a44-472e-4908-b28d-7610cc3fba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.574115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8a44-472e-4908-b28d-7610cc3fba78 admin admin] PATH_INFO: `/v3/projects/bc408ec1d22941f386c8033af38856e7/users/f7ed39d4b5d449ceb757377ffca01af9/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.600877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e8a44-472e-4908-b28d-7610cc3fba78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc408ec1d22941f386c8033af38856e7, user_id=f7ed39d4b5d449ceb757377ffca01af9, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.603757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e8a44-472e-4908-b28d-7610cc3fba78 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.620187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1312/2616] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:36 2022] PUT /identity/v3/projects/bc408ec1d22941f386c8033af38856e7/users/f7ed39d4b5d449ceb757377ffca01af9/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:36.622642 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e8a44-472e-4908-b28d-7610cc3fba78 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.624800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57f619-f267-44ec-a552-499ed25aa6df admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.698006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e8a44-472e-4908-b28d-7610cc3fba78 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.704542 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.706144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29c58d-8e87-4a78-a3ab-86066fd9346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.706437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.706749 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.706876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29c58d-8e87-4a78-a3ab-86066fd9346b admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.706993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.707089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29c58d-8e87-4a78-a3ab-86066fd9346b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.707316 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29c58d-8e87-4a78-a3ab-86066fd9346b admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.708036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29c58d-8e87-4a78-a3ab-86066fd9346b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.708947 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.709819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29c58d-8e87-4a78-a3ab-86066fd9346b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.721127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1313/2617] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:36.721986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29c58d-8e87-4a78-a3ab-86066fd9346b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.734476 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29c58d-8e87-4a78-a3ab-86066fd9346b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.741604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d5b23-ed08-4071-b3ac-ef3e52809190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.742312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d5b23-ed08-4071-b3ac-ef3e52809190 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.742596 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d5b23-ed08-4071-b3ac-ef3e52809190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.742870 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d5b23-ed08-4071-b3ac-ef3e52809190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.743518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d5b23-ed08-4071-b3ac-ef3e52809190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.745318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d5b23-ed08-4071-b3ac-ef3e52809190 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.757666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1314/2618] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:36.758677 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d5b23-ed08-4071-b3ac-ef3e52809190 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.770892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d5b23-ed08-4071-b3ac-ef3e52809190 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.775980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea23fb-2f93-414f-acbf-52660237a497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.776731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea23fb-2f93-414f-acbf-52660237a497 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.777010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea23fb-2f93-414f-acbf-52660237a497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.777279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea23fb-2f93-414f-acbf-52660237a497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.777922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea23fb-2f93-414f-acbf-52660237a497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.779702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea23fb-2f93-414f-acbf-52660237a497 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.783447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:36.792649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.794134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.797902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1315/2619] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:36.799221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea23fb-2f93-414f-acbf-52660237a497 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.807975 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1305/2620] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:36.808959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:36.810011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea23fb-2f93-414f-acbf-52660237a497 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.816582 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91cf38-d1aa-4072-ad0d-4b647d811fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.817445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91cf38-d1aa-4072-ad0d-4b647d811fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.818100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91cf38-d1aa-4072-ad0d-4b647d811fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.818603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91cf38-d1aa-4072-ad0d-4b647d811fc5 admin admin] PATH_INFO: `/v3/projects/bc408ec1d22941f386c8033af38856e7/users/f7ed39d4b5d449ceb757377ffca01af9/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.821420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba58fc-feb7-4ac3-bf0a-c2145b9d0555 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:36.821865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91cf38-d1aa-4072-ad0d-4b647d811fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc408ec1d22941f386c8033af38856e7, user_id=f7ed39d4b5d449ceb757377ffca01af9, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.827524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91cf38-d1aa-4072-ad0d-4b647d811fc5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.828944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d496a5-6636-4645-84ae-988cf6bb3c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:36.832392 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d496a5-6636-4645-84ae-988cf6bb3c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.833092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d496a5-6636-4645-84ae-988cf6bb3c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.833830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d496a5-6636-4645-84ae-988cf6bb3c8b admin admin] PATH_INFO: `/v3/projects/7cfc50ac33174447af8f4a7ecc6cc125/users/4f62872ebbfd471b9db50bcc401a1768/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.849927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1316/2621] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:36 2022] PUT /identity/v3/projects/bc408ec1d22941f386c8033af38856e7/users/f7ed39d4b5d449ceb757377ffca01af9/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:36.851692 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6949-13cd-468b-9a45-4695587061ac None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.852034 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6949-13cd-468b-9a45-4695587061ac None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.852587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6949-13cd-468b-9a45-4695587061ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.853696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d496a5-6636-4645-84ae-988cf6bb3c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cfc50ac33174447af8f4a7ecc6cc125, user_id=4f62872ebbfd471b9db50bcc401a1768, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:36.856828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d496a5-6636-4645-84ae-988cf6bb3c8b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:36.875986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1306/2622] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:36 2022] PUT /identity/v3/projects/7cfc50ac33174447af8f4a7ecc6cc125/users/4f62872ebbfd471b9db50bcc401a1768/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:36.882131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b30bf-2364-4a5f-89bd-e4e4a6f39120 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:36.882455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b30bf-2364-4a5f-89bd-e4e4a6f39120 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:36.882706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b30bf-2364-4a5f-89bd-e4e4a6f39120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:36.910645 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-907b30bf-2364-4a5f-89bd-e4e4a6f39120 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:36.912579 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-907b30bf-2364-4a5f-89bd-e4e4a6f39120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.922211 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3e4e6949-13cd-468b-9a45-4695587061ac None None] MFA Rules not processed for user `f7ed39d4b5d449ceb757377ffca01af9`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:36.923230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3e4e6949-13cd-468b-9a45-4695587061ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:36.938977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:37.023896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1307/2623] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:03:37.031454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1317/2624] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:03:37.033642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b30bf-2364-4a5f-89bd-e4e4a6f39120 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:37.037231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-907b30bf-2364-4a5f-89bd-e4e4a6f39120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:37.088654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e6949-13cd-468b-9a45-4695587061ac None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:37.090204 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3e4e6949-13cd-468b-9a45-4695587061ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:37.119373 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b30bf-2364-4a5f-89bd-e4e4a6f39120 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:37.125439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444054e-0cb9-444b-b134-50927a8d0e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:37.126535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444054e-0cb9-444b-b134-50927a8d0e4e service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:37.127700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444054e-0cb9-444b-b134-50927a8d0e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:37.128338 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444054e-0cb9-444b-b134-50927a8d0e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:37.138912 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444054e-0cb9-444b-b134-50927a8d0e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:37.140668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444054e-0cb9-444b-b134-50927a8d0e4e service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:37.153001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1308/2625] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:37.168058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e6949-13cd-468b-9a45-4695587061ac None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:37.173733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117482c7-082c-4e81-b032-7d465d162888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:37.175068 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117482c7-082c-4e81-b032-7d465d162888 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:37.175601 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117482c7-082c-4e81-b032-7d465d162888 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:37.175867 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117482c7-082c-4e81-b032-7d465d162888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:37.191262 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117482c7-082c-4e81-b032-7d465d162888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:37.192762 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117482c7-082c-4e81-b032-7d465d162888 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:37.224189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1318/2626] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:03:37.310927 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444054e-0cb9-444b-b134-50927a8d0e4e service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:37.321125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444054e-0cb9-444b-b134-50927a8d0e4e service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:37.325508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc048f0c-0362-4e73-ad77-d248d97d38a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:37.326192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc048f0c-0362-4e73-ad77-d248d97d38a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:37.326541 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc048f0c-0362-4e73-ad77-d248d97d38a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:37.326875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc048f0c-0362-4e73-ad77-d248d97d38a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:37.390091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc048f0c-0362-4e73-ad77-d248d97d38a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:37.391810 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc048f0c-0362-4e73-ad77-d248d97d38a3 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:37.402559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1309/2627] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:41.687099 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73326f3-a9d5-412e-9f5f-f66b56ac3d71 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:41.687586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73326f3-a9d5-412e-9f5f-f66b56ac3d71 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:41.687710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73326f3-a9d5-412e-9f5f-f66b56ac3d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:41.719018 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-d73326f3-a9d5-412e-9f5f-f66b56ac3d71 None None] MFA Rules not processed for user `4f62872ebbfd471b9db50bcc401a1768`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:41.720289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d73326f3-a9d5-412e-9f5f-f66b56ac3d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:41.795678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1319/2628] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:41.800587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc048f0c-0362-4e73-ad77-d248d97d38a3 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:41.811485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc048f0c-0362-4e73-ad77-d248d97d38a3 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:41.817049 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c33f79-8792-4d7a-8180-8bb9061ad91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:41.817685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c33f79-8792-4d7a-8180-8bb9061ad91a admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:41.817962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c33f79-8792-4d7a-8180-8bb9061ad91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:41.818229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c33f79-8792-4d7a-8180-8bb9061ad91a admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:41.818864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c33f79-8792-4d7a-8180-8bb9061ad91a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:41.820799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c33f79-8792-4d7a-8180-8bb9061ad91a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:41.848887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1310/2629] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:41 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:41.853049 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73326f3-a9d5-412e-9f5f-f66b56ac3d71 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:41.858707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:41.865664 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73326f3-a9d5-412e-9f5f-f66b56ac3d71 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:41.922229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b946293-320d-4140-8da9-4122ccc240fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:41.922828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b946293-320d-4140-8da9-4122ccc240fc admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:41.923103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b946293-320d-4140-8da9-4122ccc240fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:41.923368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b946293-320d-4140-8da9-4122ccc240fc admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:41.924085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b946293-320d-4140-8da9-4122ccc240fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:41.925997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b946293-320d-4140-8da9-4122ccc240fc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:41.947323 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1320/2630] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:41 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:41.951730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c33f79-8792-4d7a-8180-8bb9061ad91a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:41.963234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c33f79-8792-4d7a-8180-8bb9061ad91a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:41.967807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5eb59-bbbd-427f-bdb5-9550dae8aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:41.968441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5eb59-bbbd-427f-bdb5-9550dae8aca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:41.968720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5eb59-bbbd-427f-bdb5-9550dae8aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:41.968989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5eb59-bbbd-427f-bdb5-9550dae8aca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:41.969624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5eb59-bbbd-427f-bdb5-9550dae8aca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:41.971678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5eb59-bbbd-427f-bdb5-9550dae8aca4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:41.981934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1311/2631] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:41.987938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b946293-320d-4140-8da9-4122ccc240fc admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:41.998526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b946293-320d-4140-8da9-4122ccc240fc admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:42.003958 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c3946-1fc3-4714-9342-5c67058a8ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:42.004660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c3946-1fc3-4714-9342-5c67058a8ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:42.004949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c3946-1fc3-4714-9342-5c67058a8ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:42.005225 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c3946-1fc3-4714-9342-5c67058a8ee4 admin admin] PATH_INFO: `/v3/projects/d4bcfc504d224af9a6fde6c9964dae79/users/46080adfcafc4a239f2772597af6a21d/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:42.021574 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c3946-1fc3-4714-9342-5c67058a8ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4bcfc504d224af9a6fde6c9964dae79, user_id=46080adfcafc4a239f2772597af6a21d, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:42.024301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c3946-1fc3-4714-9342-5c67058a8ee4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:42.038407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1321/2632] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:41 2022] PUT /identity/v3/projects/d4bcfc504d224af9a6fde6c9964dae79/users/46080adfcafc4a239f2772597af6a21d/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:42.042334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5eb59-bbbd-427f-bdb5-9550dae8aca4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:42.052818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5eb59-bbbd-427f-bdb5-9550dae8aca4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:42.080950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:42.111493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77175f31-af3c-4f87-b2ef-6464b783179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:42.112223 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77175f31-af3c-4f87-b2ef-6464b783179f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:42.112504 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77175f31-af3c-4f87-b2ef-6464b783179f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:42.112779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77175f31-af3c-4f87-b2ef-6464b783179f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:42.113412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77175f31-af3c-4f87-b2ef-6464b783179f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:42.115008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77175f31-af3c-4f87-b2ef-6464b783179f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:42.124555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1312/2633] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:42.129436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c3946-1fc3-4714-9342-5c67058a8ee4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:42.139537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c3946-1fc3-4714-9342-5c67058a8ee4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:42.144782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc57065-4fc6-4030-bcbe-c100997e3a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:42.145437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57065-4fc6-4030-bcbe-c100997e3a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:42.145718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57065-4fc6-4030-bcbe-c100997e3a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:42.145993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc57065-4fc6-4030-bcbe-c100997e3a81 admin admin] PATH_INFO: `/v3/projects/d4bcfc504d224af9a6fde6c9964dae79/users/46080adfcafc4a239f2772597af6a21d/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:42.147921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc57065-4fc6-4030-bcbe-c100997e3a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4bcfc504d224af9a6fde6c9964dae79, user_id=46080adfcafc4a239f2772597af6a21d, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:42.150573 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc57065-4fc6-4030-bcbe-c100997e3a81 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:42.161006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1322/2634] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:42 2022] PUT /identity/v3/projects/d4bcfc504d224af9a6fde6c9964dae79/users/46080adfcafc4a239f2772597af6a21d/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:43.878429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77175f31-af3c-4f87-b2ef-6464b783179f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:43.888937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77175f31-af3c-4f87-b2ef-6464b783179f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:43.962568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:43.963102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:43.964586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:43.965434 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] PATH_INFO: `/v3/users/a248a53e6b6b43fe9f4cf9547a939bc7` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:43.967971 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a248a53e6b6b43fe9f4cf9547a939bc7)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:43.972949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:44.044436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a248a53e6b6b43fe9f4cf9547a939bc7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:44.056552 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a248a53e6b6b43fe9f4cf9547a939bc7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:44.057654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a248a53e6b6b43fe9f4cf9547a939bc7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:44.085515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a248a53e6b6b43fe9f4cf9547a939bc7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:44.086495 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a248a53e6b6b43fe9f4cf9547a939bc7'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:44.104718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1313/2635] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:43 2022] DELETE /identity/v3/users/a248a53e6b6b43fe9f4cf9547a939bc7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:44.109492 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad690b-2552-4227-985d-cd500fe9a44e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:44.110379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad690b-2552-4227-985d-cd500fe9a44e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:44.110854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad690b-2552-4227-985d-cd500fe9a44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:44.137517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-faad690b-2552-4227-985d-cd500fe9a44e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:44.138737 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-faad690b-2552-4227-985d-cd500fe9a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:44.212852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1323/2636] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:44.223050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:44.224767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:44.348284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:44.374633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95c550-70ef-4228-b2a4-7bd775fbbd68 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:44.397872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f53e2-b792-446b-9d2b-2ea85f99daaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:44.398476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f53e2-b792-446b-9d2b-2ea85f99daaa service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:44.401344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f53e2-b792-446b-9d2b-2ea85f99daaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:44.401344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f53e2-b792-446b-9d2b-2ea85f99daaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:44.415475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f53e2-b792-446b-9d2b-2ea85f99daaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:44.417614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f53e2-b792-446b-9d2b-2ea85f99daaa service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:44.436140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1314/2637] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:45.159048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad690b-2552-4227-985d-cd500fe9a44e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:45.161484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-faad690b-2552-4227-985d-cd500fe9a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:45.194110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad690b-2552-4227-985d-cd500fe9a44e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:45.209360 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:45.247849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:45.250456 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:45.254241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] PATH_INFO: `/v3/projects/7e001a3e5ab54b6180868a9a557a035e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:45.259185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e001a3e5ab54b6180868a9a557a035e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:45.270864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:45.461428 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e001a3e5ab54b6180868a9a557a035e', 'request_id': 'req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.465333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:45.469931 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e001a3e5ab54b6180868a9a557a035e', 'request_id': 'req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.485005 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e001a3e5ab54b6180868a9a557a035e', 'request_id': 'req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.503752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] The token cache is being invalidate because project 7e001a3e5ab54b6180868a9a557a035e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:45.504505 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.505495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.507758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1324/2638] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:45 2022] DELETE /identity/v3/projects/7e001a3e5ab54b6180868a9a557a035e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:45.512788 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f53e2-b792-446b-9d2b-2ea85f99daaa service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:45.530941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f53e2-b792-446b-9d2b-2ea85f99daaa service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:45.701364 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:45.702021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:45.702468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:45.702746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] PATH_INFO: `/v3/users/6fe0c17a6d7342deb62e7bc86715478f` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:45.704314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe0c17a6d7342deb62e7bc86715478f)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:45.706853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:45.796250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe0c17a6d7342deb62e7bc86715478f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.806583 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe0c17a6d7342deb62e7bc86715478f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.808466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe0c17a6d7342deb62e7bc86715478f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.830803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe0c17a6d7342deb62e7bc86715478f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.831546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-5f2084a4-9e0a-4cdb-ad16-ef8f02ef7be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe0c17a6d7342deb62e7bc86715478f'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:45.851464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1315/2639] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:45 2022] DELETE /identity/v3/users/6fe0c17a6d7342deb62e7bc86715478f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:46.469534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:46.472332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:46.497468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d172a-f8bc-41fd-8735-eef27cea6eeb admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:46.677471 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:46.678420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:46.678918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:46.679423 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] PATH_INFO: `/v3/projects/014a1b871a6b4546aa4bd05b66aba241` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:46.681187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=014a1b871a6b4546aa4bd05b66aba241)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:46.685712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:46.781269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014a1b871a6b4546aa4bd05b66aba241', 'request_id': 'req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:46.810683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014a1b871a6b4546aa4bd05b66aba241', 'request_id': 'req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:46.828432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014a1b871a6b4546aa4bd05b66aba241', 'request_id': 'req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:46.851538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] The token cache is being invalidate because project 014a1b871a6b4546aa4bd05b66aba241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:46.851842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:46.853466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-68393c2f-eb55-405a-ae8a-a3c9f6cdf136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:46.868401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1325/2640] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:46 2022] DELETE /identity/v3/projects/014a1b871a6b4546aa4bd05b66aba241 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:46.868543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44c65d-61f9-4922-8c0d-6df2904ca88c None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:46.868543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44c65d-61f9-4922-8c0d-6df2904ca88c None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:46.868543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44c65d-61f9-4922-8c0d-6df2904ca88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:46.904955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-de44c65d-61f9-4922-8c0d-6df2904ca88c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:46.906221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-de44c65d-61f9-4922-8c0d-6df2904ca88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:47.040229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1316/2641] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:47.050268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743224a3-f552-42fb-bac6-135c2f10cb60 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:47.052559 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743224a3-f552-42fb-bac6-135c2f10cb60 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:47.053017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743224a3-f552-42fb-bac6-135c2f10cb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:47.097875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-743224a3-f552-42fb-bac6-135c2f10cb60 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:47.101785 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-743224a3-f552-42fb-bac6-135c2f10cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:47.120859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1326/2642] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:47.124883 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44c65d-61f9-4922-8c0d-6df2904ca88c None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:47.140538 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44c65d-61f9-4922-8c0d-6df2904ca88c None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:47.153825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:47.154464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:47.154746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:47.155020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:47.155692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:03:47.155999 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:47.157649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:47.160354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:47.207352 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1317/2643] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:03:47.208736 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743224a3-f552-42fb-bac6-135c2f10cb60 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:47.223674 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743224a3-f552-42fb-bac6-135c2f10cb60 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:47.239992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684370a2-f51f-4897-a999-244de9f2c30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:47.241003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684370a2-f51f-4897-a999-244de9f2c30c admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:47.241402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684370a2-f51f-4897-a999-244de9f2c30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:47.241778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684370a2-f51f-4897-a999-244de9f2c30c admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:47.242548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684370a2-f51f-4897-a999-244de9f2c30c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:47.245334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684370a2-f51f-4897-a999-244de9f2c30c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:47.311004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1327/2644] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:47 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:47.315634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:47.330602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74473d76-c6da-46bb-8f08-606f59baebed admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:47.467563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c80b4-72bc-4cde-96eb-629929f87bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:47.468341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c80b4-72bc-4cde-96eb-629929f87bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:47.468623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c80b4-72bc-4cde-96eb-629929f87bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:47.468972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c80b4-72bc-4cde-96eb-629929f87bcd admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:47.471093 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c80b4-72bc-4cde-96eb-629929f87bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:47.473444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c80b4-72bc-4cde-96eb-629929f87bcd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:47.506168 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1318/2645] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:47 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:03:47.509835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684370a2-f51f-4897-a999-244de9f2c30c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:47.522047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684370a2-f51f-4897-a999-244de9f2c30c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:47.538332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd8426-629e-4475-83d5-d37edb8f676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:47.539845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd8426-629e-4475-83d5-d37edb8f676f admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:47.540208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd8426-629e-4475-83d5-d37edb8f676f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:47.540488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd8426-629e-4475-83d5-d37edb8f676f admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:47.541427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd8426-629e-4475-83d5-d37edb8f676f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:47.543479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd8426-629e-4475-83d5-d37edb8f676f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:47.557282 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1328/2646] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:47.561257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c80b4-72bc-4cde-96eb-629929f87bcd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:47.584288 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c80b4-72bc-4cde-96eb-629929f87bcd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:47.589286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d34617-9ce3-4ca0-a66b-9378ce9e1851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:47.589904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d34617-9ce3-4ca0-a66b-9378ce9e1851 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:47.590182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d34617-9ce3-4ca0-a66b-9378ce9e1851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:47.590453 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d34617-9ce3-4ca0-a66b-9378ce9e1851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:47.591100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d34617-9ce3-4ca0-a66b-9378ce9e1851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:47.592774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d34617-9ce3-4ca0-a66b-9378ce9e1851 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:47.625840 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1319/2647] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:03:47.628074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd8426-629e-4475-83d5-d37edb8f676f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:47.641307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd8426-629e-4475-83d5-d37edb8f676f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:47.646427 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:47.647353 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:47.647702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:47.648053 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] PATH_INFO: `/v3/projects/189efb65725e4112a37b2cee4b1d5c44/users/46b0dd19f2624eadae942a373bb8a656/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:47.667781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189efb65725e4112a37b2cee4b1d5c44, user_id=46b0dd19f2624eadae942a373bb8a656, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:47.670648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:47.692380 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1329/2648] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:47 2022] PUT /identity/v3/projects/189efb65725e4112a37b2cee4b1d5c44/users/46b0dd19f2624eadae942a373bb8a656/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:03:47.700744 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5bfa-13b1-4a13-b019-b94f71dd0e00 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:47.701045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5bfa-13b1-4a13-b019-b94f71dd0e00 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:47.701306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc5bfa-13b1-4a13-b019-b94f71dd0e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:47.734690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-90dc5bfa-13b1-4a13-b019-b94f71dd0e00 None None] MFA Rules not processed for user `46b0dd19f2624eadae942a373bb8a656`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:47.736273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-90dc5bfa-13b1-4a13-b019-b94f71dd0e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:47.886051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1320/2649] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:47.895032 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:47.898797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:47.907928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:48.047802 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b21f5-1286-42b4-88bf-cb9179d63534 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:48.054214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:48.054446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:48.054832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:48.055165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:48.066231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:48.067771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:48.084408 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1330/2650] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:48.140956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8003875-46a8-46e3-b8cf-e6e0d6fefe9e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:48.141240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8003875-46a8-46e3-b8cf-e6e0d6fefe9e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:48.141513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8003875-46a8-46e3-b8cf-e6e0d6fefe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:48.202869 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e8003875-46a8-46e3-b8cf-e6e0d6fefe9e None None] MFA Rules not processed for user `46080adfcafc4a239f2772597af6a21d`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:48.204293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e8003875-46a8-46e3-b8cf-e6e0d6fefe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:48.299835 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:48.340630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1321/2651] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:48.352661 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:48.354144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:48.455832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1745c-9c54-44ad-9516-e454bf0b8efe service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:48.460519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7a2237c18934cb2aa3699a5776874ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:48.473723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:48.474193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:48.476148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:48.479441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:48.592930 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:48.595272 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:48.608070 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1331/2652] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:48.750691 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508d80-ce88-4153-aac1-1c3bf31c24aa None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:48.750979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508d80-ce88-4153-aac1-1c3bf31c24aa None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:48.751257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508d80-ce88-4153-aac1-1c3bf31c24aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:48.815370 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6c508d80-ce88-4153-aac1-1c3bf31c24aa None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:48.816568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6c508d80-ce88-4153-aac1-1c3bf31c24aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:48.885925 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1322/2653] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 17:03:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:48.938896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:48.940620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:49.018668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:49.046844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bc0f1-01a3-4f1c-81fc-23948c0d3b9c service glance] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:49.053164 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80b63f-c27d-4a64-bee3-1bdc921f4c0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:49.053771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80b63f-c27d-4a64-bee3-1bdc921f4c0c service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:49.054048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80b63f-c27d-4a64-bee3-1bdc921f4c0c service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:49.054320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80b63f-c27d-4a64-bee3-1bdc921f4c0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:49.062545 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca80b63f-c27d-4a64-bee3-1bdc921f4c0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:49.064183 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca80b63f-c27d-4a64-bee3-1bdc921f4c0c service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:49.072772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1332/2654] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 17:03:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 17:03:51.882054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c508d80-ce88-4153-aac1-1c3bf31c24aa None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:51.986867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c508d80-ce88-4153-aac1-1c3bf31c24aa None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:51.994498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:51.995350 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:51.995799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:51.996604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] PATH_INFO: `/v3/users/46b0dd19f2624eadae942a373bb8a656` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:51.998218 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b0dd19f2624eadae942a373bb8a656)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:52.000624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:52.074861 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b0dd19f2624eadae942a373bb8a656'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:52.081619 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:52.088134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b0dd19f2624eadae942a373bb8a656'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:52.089463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b0dd19f2624eadae942a373bb8a656'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:52.113435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b0dd19f2624eadae942a373bb8a656'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:52.115276 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b0dd19f2624eadae942a373bb8a656'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:52.138118 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1323/2655] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:51 2022] DELETE /identity/v3/users/46b0dd19f2624eadae942a373bb8a656 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:52.142668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e8dbc-fbdd-45f5-8733-2a559c918f46 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:52.143170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e8dbc-fbdd-45f5-8733-2a559c918f46 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:52.143455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e8dbc-fbdd-45f5-8733-2a559c918f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:52.164747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4d3e8dbc-fbdd-45f5-8733-2a559c918f46 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:52.166474 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4d3e8dbc-fbdd-45f5-8733-2a559c918f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:52.251275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1333/2656] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:52.259428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:52.262033 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:52.438742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f5162-de1e-4f8c-b26c-e23911f6d834 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:52.453336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:52.454051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:52.454533 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:52.455073 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:52.481224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:52.484750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:52.498139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1324/2657] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:53.137244 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e8dbc-fbdd-45f5-8733-2a559c918f46 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:53.138774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4d3e8dbc-fbdd-45f5-8733-2a559c918f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:53.188630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e8dbc-fbdd-45f5-8733-2a559c918f46 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:53.198973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:53.199946 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:53.200829 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:53.201119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] PATH_INFO: `/v3/projects/189efb65725e4112a37b2cee4b1d5c44` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:53.203532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=189efb65725e4112a37b2cee4b1d5c44)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:53.208732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:53.291165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189efb65725e4112a37b2cee4b1d5c44', 'request_id': 'req-47a820ec-0ed7-4249-9639-9e7c8d2ea870'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:53.302007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189efb65725e4112a37b2cee4b1d5c44', 'request_id': 'req-47a820ec-0ed7-4249-9639-9e7c8d2ea870'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:53.317019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189efb65725e4112a37b2cee4b1d5c44', 'request_id': 'req-47a820ec-0ed7-4249-9639-9e7c8d2ea870'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:53.345291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] The token cache is being invalidate because project 189efb65725e4112a37b2cee4b1d5c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:53.346539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:53.347510 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-47a820ec-0ed7-4249-9639-9e7c8d2ea870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:53.364159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1334/2658] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:53 2022] DELETE /identity/v3/projects/189efb65725e4112a37b2cee4b1d5c44 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:54.154109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:54.155643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:54.178282 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dab144-3993-4fae-b92f-d1af5a9dac6c service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:54.254253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:54.255226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:54.255614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:54.255993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] PATH_INFO: `/v3/users/e0accb8344df4cd1b3959212b654f66b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:54.257535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0accb8344df4cd1b3959212b654f66b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:54.259572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:54.320795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0accb8344df4cd1b3959212b654f66b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:54.331508 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0accb8344df4cd1b3959212b654f66b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:54.332507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0accb8344df4cd1b3959212b654f66b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:54.347844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0accb8344df4cd1b3959212b654f66b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:54.348881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0accb8344df4cd1b3959212b654f66b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:54.351404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:54.372359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1325/2659] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:54 2022] DELETE /identity/v3/users/e0accb8344df4cd1b3959212b654f66b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:54.377814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfeb6ab-6e7b-427f-bc63-1b6fe97af37f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:54.378450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfeb6ab-6e7b-427f-bc63-1b6fe97af37f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:54.378578 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfeb6ab-6e7b-427f-bc63-1b6fe97af37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:54.404758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4dfeb6ab-6e7b-427f-bc63-1b6fe97af37f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:54.406971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4dfeb6ab-6e7b-427f-bc63-1b6fe97af37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:54.613919 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:54.629587 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1335/2660] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:03:54.639525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:54.640980 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:54.716475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187f64e-0ead-41cf-a40e-923b34a360f7 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:54.720228 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e48fa-2d9d-4549-a0a7-7ccd1465713f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:54.721097 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e48fa-2d9d-4549-a0a7-7ccd1465713f service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:54.721374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e48fa-2d9d-4549-a0a7-7ccd1465713f service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:54.721644 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e48fa-2d9d-4549-a0a7-7ccd1465713f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:54.733610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e48fa-2d9d-4549-a0a7-7ccd1465713f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:54.735197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e48fa-2d9d-4549-a0a7-7ccd1465713f service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:54.749257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1326/2661] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:55.060079 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfeb6ab-6e7b-427f-bc63-1b6fe97af37f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:55.061549 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4dfeb6ab-6e7b-427f-bc63-1b6fe97af37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:55.077341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfeb6ab-6e7b-427f-bc63-1b6fe97af37f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:55.084170 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:55.084979 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:55.085308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:55.085645 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] PATH_INFO: `/v3/projects/6432e6a1173b4fec9083fb94936b069e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:55.086896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6432e6a1173b4fec9083fb94936b069e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:55.088963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:55.147694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6432e6a1173b4fec9083fb94936b069e', 'request_id': 'req-1a12f211-1c9f-4b67-9c36-63aa8a634482'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.154663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6432e6a1173b4fec9083fb94936b069e', 'request_id': 'req-1a12f211-1c9f-4b67-9c36-63aa8a634482'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.161151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6432e6a1173b4fec9083fb94936b069e', 'request_id': 'req-1a12f211-1c9f-4b67-9c36-63aa8a634482'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.180825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] The token cache is being invalidate because project 6432e6a1173b4fec9083fb94936b069e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:55.181108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.181846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.183865 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1336/2662] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:55 2022] DELETE /identity/v3/projects/6432e6a1173b4fec9083fb94936b069e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:55.190301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e48fa-2d9d-4549-a0a7-7ccd1465713f service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:55.201262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e48fa-2d9d-4549-a0a7-7ccd1465713f service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:55.291983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:55.292659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:55.292936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:55.293204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] PATH_INFO: `/v3/users/f7ed39d4b5d449ceb757377ffca01af9` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:55.294354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ed39d4b5d449ceb757377ffca01af9)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:55.296372 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:55.351412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ed39d4b5d449ceb757377ffca01af9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.363445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ed39d4b5d449ceb757377ffca01af9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.364361 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ed39d4b5d449ceb757377ffca01af9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.381077 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ed39d4b5d449ceb757377ffca01af9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.381816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-08c54ec9-ade6-4bcb-b8bc-d25901a2895a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ed39d4b5d449ceb757377ffca01af9'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.396819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1327/2663] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:03:55 2022] DELETE /identity/v3/users/f7ed39d4b5d449ceb757377ffca01af9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:55.748124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:55.749744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:55.765142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12f211-1c9f-4b67-9c36-63aa8a634482 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:55.849497 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:55.850248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:55.850705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:55.851038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] PATH_INFO: `/v3/projects/bc408ec1d22941f386c8033af38856e7` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:55.852650 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc408ec1d22941f386c8033af38856e7)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:55.855543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:55.911718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc408ec1d22941f386c8033af38856e7', 'request_id': 'req-34188094-31d4-4c54-b3dd-a9202452c76d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.919121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc408ec1d22941f386c8033af38856e7', 'request_id': 'req-34188094-31d4-4c54-b3dd-a9202452c76d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.925523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc408ec1d22941f386c8033af38856e7', 'request_id': 'req-34188094-31d4-4c54-b3dd-a9202452c76d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.945776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] The token cache is being invalidate because project bc408ec1d22941f386c8033af38856e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:03:55.946069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.946748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-34188094-31d4-4c54-b3dd-a9202452c76d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:03:55.948543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1337/2664] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:03:55 2022] DELETE /identity/v3/projects/bc408ec1d22941f386c8033af38856e7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:55.955480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eda36-2674-49c6-bc82-e408cfc6f216 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:55.955956 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eda36-2674-49c6-bc82-e408cfc6f216 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:55.956254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2eda36-2674-49c6-bc82-e408cfc6f216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:55.977339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ba2eda36-2674-49c6-bc82-e408cfc6f216 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:55.978574 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ba2eda36-2674-49c6-bc82-e408cfc6f216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:55.989283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:56.054950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1328/2665] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:56.063094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277ecaf-1a72-4680-b29f-f3ddeaded073 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.063461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277ecaf-1a72-4680-b29f-f3ddeaded073 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.063791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277ecaf-1a72-4680-b29f-f3ddeaded073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.085817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e277ecaf-1a72-4680-b29f-f3ddeaded073 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:56.087048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e277ecaf-1a72-4680-b29f-f3ddeaded073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:56.095228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1338/2666] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:56.100156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2eda36-2674-49c6-bc82-e408cfc6f216 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.110469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2eda36-2674-49c6-bc82-e408cfc6f216 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.115408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.116034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.116340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.116613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.117467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:03:56.117773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.119394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.135860 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1329/2667] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:03:56.140542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277ecaf-1a72-4680-b29f-f3ddeaded073 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.145566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:56.151973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277ecaf-1a72-4680-b29f-f3ddeaded073 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.159477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2f28-308a-46c3-bd0a-4863170c0688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.160404 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2f28-308a-46c3-bd0a-4863170c0688 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.160735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2f28-308a-46c3-bd0a-4863170c0688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.161006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f2f28-308a-46c3-bd0a-4863170c0688 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.161682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f2f28-308a-46c3-bd0a-4863170c0688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.163638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f2f28-308a-46c3-bd0a-4863170c0688 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.198265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1339/2668] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:56 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:56.203211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.222192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa639c-07f8-486e-a67c-8655f120a785 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.307142 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235f387-593f-4817-b8f3-bf0d11b95b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.307762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235f387-593f-4817-b8f3-bf0d11b95b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.308050 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235f387-593f-4817-b8f3-bf0d11b95b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.308354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235f387-593f-4817-b8f3-bf0d11b95b4b admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.309515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235f387-593f-4817-b8f3-bf0d11b95b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.311571 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235f387-593f-4817-b8f3-bf0d11b95b4b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.335813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1330/2669] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:56 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:56.340134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2f28-308a-46c3-bd0a-4863170c0688 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.353098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f2f28-308a-46c3-bd0a-4863170c0688 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.360783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d129e8-f430-47d4-ad64-97d85b4f1e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.361620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d129e8-f430-47d4-ad64-97d85b4f1e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.361620 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d129e8-f430-47d4-ad64-97d85b4f1e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.361834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d129e8-f430-47d4-ad64-97d85b4f1e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.362748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d129e8-f430-47d4-ad64-97d85b4f1e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.364468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d129e8-f430-47d4-ad64-97d85b4f1e5a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.376187 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1340/2670] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:56.380877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235f387-593f-4817-b8f3-bf0d11b95b4b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.394858 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235f387-593f-4817-b8f3-bf0d11b95b4b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.399640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69717efc-86d2-4a56-8ae5-22847755dedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.400405 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69717efc-86d2-4a56-8ae5-22847755dedb admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.401188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69717efc-86d2-4a56-8ae5-22847755dedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.401188 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69717efc-86d2-4a56-8ae5-22847755dedb admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.401711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69717efc-86d2-4a56-8ae5-22847755dedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.403333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69717efc-86d2-4a56-8ae5-22847755dedb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.415058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1331/2671] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:56.419259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d129e8-f430-47d4-ad64-97d85b4f1e5a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.433580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d129e8-f430-47d4-ad64-97d85b4f1e5a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.438335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caeedc-ab91-4a8a-a7e7-6b9096166557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.438992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caeedc-ab91-4a8a-a7e7-6b9096166557 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.440546 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caeedc-ab91-4a8a-a7e7-6b9096166557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.440830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19caeedc-ab91-4a8a-a7e7-6b9096166557 admin admin] PATH_INFO: `/v3/projects/117062957b024a448e3ddf5db8662a49/users/83b771b330bf418099b26755b1ea1f92/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.458495 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19caeedc-ab91-4a8a-a7e7-6b9096166557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=117062957b024a448e3ddf5db8662a49, user_id=83b771b330bf418099b26755b1ea1f92, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.465340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19caeedc-ab91-4a8a-a7e7-6b9096166557 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.479901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1341/2672] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:56 2022] PUT /identity/v3/projects/117062957b024a448e3ddf5db8662a49/users/83b771b330bf418099b26755b1ea1f92/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:56.490503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ca6f6-601c-40c6-be2f-d307da5675a0 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.490787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ca6f6-601c-40c6-be2f-d307da5675a0 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.491071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ca6f6-601c-40c6-be2f-d307da5675a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.531374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-ab7ca6f6-601c-40c6-be2f-d307da5675a0 None None] MFA Rules not processed for user `83b771b330bf418099b26755b1ea1f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:56.533318 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ab7ca6f6-601c-40c6-be2f-d307da5675a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:56.644002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1332/2673] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:56.648566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caeedc-ab91-4a8a-a7e7-6b9096166557 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.735817 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19caeedc-ab91-4a8a-a7e7-6b9096166557 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.740488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91e0f-376c-4c4e-8801-4491bb03e9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.741410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91e0f-376c-4c4e-8801-4491bb03e9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.741765 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91e0f-376c-4c4e-8801-4491bb03e9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.742113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d91e0f-376c-4c4e-8801-4491bb03e9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.742815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d91e0f-376c-4c4e-8801-4491bb03e9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.744806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d91e0f-376c-4c4e-8801-4491bb03e9f7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.778826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1342/2674] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:03:56 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:03:56.782468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ca6f6-601c-40c6-be2f-d307da5675a0 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.796181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ca6f6-601c-40c6-be2f-d307da5675a0 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.802599 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:56.865706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217fc64-d296-4f2f-9064-167d76286e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.865706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fc64-d296-4f2f-9064-167d76286e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.865706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fc64-d296-4f2f-9064-167d76286e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.865706 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217fc64-d296-4f2f-9064-167d76286e34 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.867667 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217fc64-d296-4f2f-9064-167d76286e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.868595 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217fc64-d296-4f2f-9064-167d76286e34 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.897001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1333/2675] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:03:56 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:03:56.900638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91e0f-376c-4c4e-8801-4491bb03e9f7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.912072 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d91e0f-376c-4c4e-8801-4491bb03e9f7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.918868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31dfe1-68ef-4dbe-88db-23d467a60718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.918868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31dfe1-68ef-4dbe-88db-23d467a60718 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.918868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31dfe1-68ef-4dbe-88db-23d467a60718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.918868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31dfe1-68ef-4dbe-88db-23d467a60718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.919365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31dfe1-68ef-4dbe-88db-23d467a60718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.920480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31dfe1-68ef-4dbe-88db-23d467a60718 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.931255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1343/2676] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:03:56.935443 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217fc64-d296-4f2f-9064-167d76286e34 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:56.946024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217fc64-d296-4f2f-9064-167d76286e34 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:56.951231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b3266-46cd-4ec0-916b-ed3c30162b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:56.951965 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b3266-46cd-4ec0-916b-ed3c30162b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:56.952351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b3266-46cd-4ec0-916b-ed3c30162b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:56.952694 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b3266-46cd-4ec0-916b-ed3c30162b9c admin admin] PATH_INFO: `/v3/projects/0c158bcd6d2f4a43ac4eb8805788451e/users/25b887a7bda045ebb317301e4be6366a/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:56.972634 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b3266-46cd-4ec0-916b-ed3c30162b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c158bcd6d2f4a43ac4eb8805788451e, user_id=25b887a7bda045ebb317301e4be6366a, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:56.975445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b3266-46cd-4ec0-916b-ed3c30162b9c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:56.995493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1334/2677] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:56 2022] PUT /identity/v3/projects/0c158bcd6d2f4a43ac4eb8805788451e/users/25b887a7bda045ebb317301e4be6366a/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:56.996734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31dfe1-68ef-4dbe-88db-23d467a60718 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:57.007227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31dfe1-68ef-4dbe-88db-23d467a60718 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:57.086425 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0e5c9-b4ab-447a-99af-05dfbb706c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:57.087153 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e5c9-b4ab-447a-99af-05dfbb706c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:57.087496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e5c9-b4ab-447a-99af-05dfbb706c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:57.087821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e5c9-b4ab-447a-99af-05dfbb706c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:57.088774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0e5c9-b4ab-447a-99af-05dfbb706c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:57.090422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0e5c9-b4ab-447a-99af-05dfbb706c9b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:57.105165 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1344/2678] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:03:57.106442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b3266-46cd-4ec0-916b-ed3c30162b9c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:57.118558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b3266-46cd-4ec0-916b-ed3c30162b9c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:57.123478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:57.124465 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:57.124833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:57.125182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] PATH_INFO: `/v3/projects/0c158bcd6d2f4a43ac4eb8805788451e/users/25b887a7bda045ebb317301e4be6366a/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:57.127789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c158bcd6d2f4a43ac4eb8805788451e, user_id=25b887a7bda045ebb317301e4be6366a, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:57.130592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:57.141464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1335/2679] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:03:57 2022] PUT /identity/v3/projects/0c158bcd6d2f4a43ac4eb8805788451e/users/25b887a7bda045ebb317301e4be6366a/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:03:57.210809 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2a5b-bb6d-4c7c-81ff-d34d381de314 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:57.211098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2a5b-bb6d-4c7c-81ff-d34d381de314 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:57.211376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2a5b-bb6d-4c7c-81ff-d34d381de314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:57.247898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-a8af2a5b-bb6d-4c7c-81ff-d34d381de314 None None] MFA Rules not processed for user `25b887a7bda045ebb317301e4be6366a`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:57.249150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a8af2a5b-bb6d-4c7c-81ff-d34d381de314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:57.253897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:03:57.340266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1345/2680] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:57.353967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:57.354594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:57.427243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4753b5b-c3e8-4d07-b624-4013f6404ab8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:57.431817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025feff-81bc-4ee3-bb38-c5565aad24ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:57.432481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025feff-81bc-4ee3-bb38-c5565aad24ef service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:57.432904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025feff-81bc-4ee3-bb38-c5565aad24ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:57.433182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025feff-81bc-4ee3-bb38-c5565aad24ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:57.445521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025feff-81bc-4ee3-bb38-c5565aad24ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:57.446988 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025feff-81bc-4ee3-bb38-c5565aad24ef service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:57.457494 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1336/2681] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:57.574006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af2a5b-bb6d-4c7c-81ff-d34d381de314 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:57.586831 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af2a5b-bb6d-4c7c-81ff-d34d381de314 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:57.590403 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:57.591000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:57.591279 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:57.591552 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:57.687003 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:57.688502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:57.700347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1346/2682] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:03:59.494121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47038c5-1026-476d-be09-53510da7eff7 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:59.494617 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47038c5-1026-476d-be09-53510da7eff7 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:59.494875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47038c5-1026-476d-be09-53510da7eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:59.516755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d47038c5-1026-476d-be09-53510da7eff7 None None] MFA Rules not processed for user `4f62872ebbfd471b9db50bcc401a1768`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:03:59.518234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d47038c5-1026-476d-be09-53510da7eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:59.580878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1337/2683] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:03:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:03:59.845234 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:03:59.847714 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:59.908749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5128f4a-7782-4dc2-8baf-ab760c023632 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:03:59.912358 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:03:59.912989 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:03:59.913269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:03:59.913542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:03:59.915281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:03:59.924702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:03:59.926124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:03:59.934318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1347/2684] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:03:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:00.300990 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47038c5-1026-476d-be09-53510da7eff7 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:00.302400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d47038c5-1026-476d-be09-53510da7eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:00.313623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:00.385181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47038c5-1026-476d-be09-53510da7eff7 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:00.390180 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653277c6-67a0-498f-9066-af979c5d779c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:00.390795 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653277c6-67a0-498f-9066-af979c5d779c service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:00.391071 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653277c6-67a0-498f-9066-af979c5d779c service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:00.391341 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653277c6-67a0-498f-9066-af979c5d779c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:00.393827 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-653277c6-67a0-498f-9066-af979c5d779c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:00.406862 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653277c6-67a0-498f-9066-af979c5d779c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:00.409640 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653277c6-67a0-498f-9066-af979c5d779c service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:00.422611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1338/2685] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:01.459120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:01.508511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:01.529420 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24718850-f14a-434a-8814-7fd15fa35549 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:01.534862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:01.535494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:01.535769 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:01.536038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] PATH_INFO: `/v3/users/83b771b330bf418099b26755b1ea1f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:01.537464 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b771b330bf418099b26755b1ea1f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:01.539387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:01.600342 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b771b330bf418099b26755b1ea1f92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:01.613579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b771b330bf418099b26755b1ea1f92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:01.614281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b771b330bf418099b26755b1ea1f92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:01.621348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b771b330bf418099b26755b1ea1f92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:01.630499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-20ca7685-874b-4838-89d5-de68f3c9ffb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b771b330bf418099b26755b1ea1f92'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:01.638142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1348/2686] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:01 2022] DELETE /identity/v3/users/83b771b330bf418099b26755b1ea1f92 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:01.642948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331149fd-9a1e-45ff-832c-49aa94e77537 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:01.643428 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331149fd-9a1e-45ff-832c-49aa94e77537 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:01.643560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331149fd-9a1e-45ff-832c-49aa94e77537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:01.663909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-331149fd-9a1e-45ff-832c-49aa94e77537 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:01.665347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-331149fd-9a1e-45ff-832c-49aa94e77537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:01.702568 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043355fe-106f-4ab0-a31f-4c81509d6237 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:01.702854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043355fe-106f-4ab0-a31f-4c81509d6237 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:01.703130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043355fe-106f-4ab0-a31f-4c81509d6237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:01.723887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-043355fe-106f-4ab0-a31f-4c81509d6237 None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:01.725157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-043355fe-106f-4ab0-a31f-4c81509d6237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:01.730598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1339/2687] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:01.742196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331149fd-9a1e-45ff-832c-49aa94e77537 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:01.743669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-331149fd-9a1e-45ff-832c-49aa94e77537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:01.791199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1349/2688] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Aug 11 17:04:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:01.804047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043355fe-106f-4ab0-a31f-4c81509d6237 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:01.805579 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-043355fe-106f-4ab0-a31f-4c81509d6237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:01.809206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331149fd-9a1e-45ff-832c-49aa94e77537 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:01.813113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:01.814458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:01.814704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:01.815030 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:01.826496 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:01.828139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:01.840232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1340/2689] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:04:01.883023 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043355fe-106f-4ab0-a31f-4c81509d6237 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:01.888615 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7e145-5738-4927-8543-81583ed0e146 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:01.889309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7e145-5738-4927-8543-81583ed0e146 service swift] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:01.889925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7e145-5738-4927-8543-81583ed0e146 service swift] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:01.889925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7e145-5738-4927-8543-81583ed0e146 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:01.896781 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:01.900122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7e145-5738-4927-8543-81583ed0e146 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:01.901940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7e145-5738-4927-8543-81583ed0e146 service swift] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:01.913550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1350/2690] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 17:04:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 11 17:04:02.387693 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:02.389543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:02.434313 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29008de8-ae7d-466d-a564-aa1645920dd3 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:02.445304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:02.447347 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:02.448035 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:02.449120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] PATH_INFO: `/v3/projects/117062957b024a448e3ddf5db8662a49` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:02.453042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=117062957b024a448e3ddf5db8662a49)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:02.474900 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:02.614442 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:02.637478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117062957b024a448e3ddf5db8662a49', 'request_id': 'req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:02.663648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117062957b024a448e3ddf5db8662a49', 'request_id': 'req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:02.693151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117062957b024a448e3ddf5db8662a49', 'request_id': 'req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:02.719784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] The token cache is being invalidate because project 117062957b024a448e3ddf5db8662a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:02.727592 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:02.740317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:02.744435 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1341/2691] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:02 2022] DELETE /identity/v3/projects/117062957b024a448e3ddf5db8662a49 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:02.749494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7e145-5738-4927-8543-81583ed0e146 service swift] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:02.771575 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7e145-5738-4927-8543-81583ed0e146 service swift] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:02.896666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:02.898415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:02.968810 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:02.972223 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:02.972864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:02.973365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] PATH_INFO: `/v3/users/25b887a7bda045ebb317301e4be6366a` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:02.977388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b887a7bda045ebb317301e4be6366a)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:02.979460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:03.009854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70d0bb-3c1d-4c3e-a44e-924ee93bcd68 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:03.022296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:03.023242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:03.023838 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:03.024311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] PATH_INFO: `/v3/users/abef050ce49f40de82a0415535e31c85` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:03.025868 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abef050ce49f40de82a0415535e31c85)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:03.028000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:03.094833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b887a7bda045ebb317301e4be6366a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.121046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abef050ce49f40de82a0415535e31c85'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.128755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b887a7bda045ebb317301e4be6366a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.129866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b887a7bda045ebb317301e4be6366a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.135637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abef050ce49f40de82a0415535e31c85'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.136431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abef050ce49f40de82a0415535e31c85'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.138481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b887a7bda045ebb317301e4be6366a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.151011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-c8cbaa90-3c9c-46f8-b441-36ce5ee82aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b887a7bda045ebb317301e4be6366a'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.160375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1351/2692] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:02 2022] DELETE /identity/v3/users/25b887a7bda045ebb317301e4be6366a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:03.170819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abef050ce49f40de82a0415535e31c85'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.171725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abef050ce49f40de82a0415535e31c85'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.199344 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1342/2693] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:02 2022] DELETE /identity/v3/users/abef050ce49f40de82a0415535e31c85 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:03.205461 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0a535-2c19-4cf2-a02a-a1dd0b7dfa5c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:03.205733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0a535-2c19-4cf2-a02a-a1dd0b7dfa5c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:03.206092 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0a535-2c19-4cf2-a02a-a1dd0b7dfa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:03.242583 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-9be0a535-2c19-4cf2-a02a-a1dd0b7dfa5c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:03.243868 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9be0a535-2c19-4cf2-a02a-a1dd0b7dfa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:03.396878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1352/2694] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:03.433205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:03.437754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:03.540536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:03.568722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b217-2040-4233-b77b-fdcd0d75d108 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:03.578206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee23114-e67d-4a84-8e37-3d93ecee3424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:03.579418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23114-e67d-4a84-8e37-3d93ecee3424 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:03.581727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23114-e67d-4a84-8e37-3d93ecee3424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:03.582008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee23114-e67d-4a84-8e37-3d93ecee3424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:03.602213 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee23114-e67d-4a84-8e37-3d93ecee3424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:03.604700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee23114-e67d-4a84-8e37-3d93ecee3424 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:03.622099 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1343/2695] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:03.786367 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0a535-2c19-4cf2-a02a-a1dd0b7dfa5c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:03.788547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-9be0a535-2c19-4cf2-a02a-a1dd0b7dfa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:03.817840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0a535-2c19-4cf2-a02a-a1dd0b7dfa5c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:03.826660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:03.827378 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:03.827701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:03.828127 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] PATH_INFO: `/v3/projects/0c158bcd6d2f4a43ac4eb8805788451e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:03.830531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c158bcd6d2f4a43ac4eb8805788451e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:03.833172 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:03.867407 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:03.961168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c158bcd6d2f4a43ac4eb8805788451e', 'request_id': 'req-678907a9-4688-487a-a95b-630b3d548186'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.970021 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c158bcd6d2f4a43ac4eb8805788451e', 'request_id': 'req-678907a9-4688-487a-a95b-630b3d548186'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:03.979320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c158bcd6d2f4a43ac4eb8805788451e', 'request_id': 'req-678907a9-4688-487a-a95b-630b3d548186'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.030878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] The token cache is being invalidate because project 0c158bcd6d2f4a43ac4eb8805788451e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:04.032356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.036654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.065151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1353/2696] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:03 2022] DELETE /identity/v3/projects/0c158bcd6d2f4a43ac4eb8805788451e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:04.068985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07483-c193-4a40-be2e-f7a754dc0f5e None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.069456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07483-c193-4a40-be2e-f7a754dc0f5e None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.069774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07483-c193-4a40-be2e-f7a754dc0f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.128577 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2ee07483-c193-4a40-be2e-f7a754dc0f5e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:04.131011 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2ee07483-c193-4a40-be2e-f7a754dc0f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:04.260181 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:04.261804 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:04.340551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1344/2697] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:04.348755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646ed9-6d57-4268-a8c0-b67fd7a7964f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.349305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646ed9-6d57-4268-a8c0-b67fd7a7964f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.349867 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c646ed9-6d57-4268-a8c0-b67fd7a7964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.440673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-2c646ed9-6d57-4268-a8c0-b67fd7a7964f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:04.451156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-2c646ed9-6d57-4268-a8c0-b67fd7a7964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:04.488032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1345/2698] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:04.496036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c646ed9-6d57-4268-a8c0-b67fd7a7964f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:04.509698 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c646ed9-6d57-4268-a8c0-b67fd7a7964f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:04.530267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:04.530267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.530267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.530830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.531602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:04.532133 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:04.534368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:04.534889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678907a9-4688-487a-a95b-630b3d548186 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:04.550499 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:04.551150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.554743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.555052 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] PATH_INFO: `/v3/projects/2e8a72152aa74463a4d321827f18dd45` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.557245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8a72152aa74463a4d321827f18dd45)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:04.561309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:04.562141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1346/2699] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:04:04.568930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:04.585737 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884d89-e099-40e7-b848-6c759b77146b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:04.595629 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd289da-5f8f-4340-bdce-f990e4fb7830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:04.597677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd289da-5f8f-4340-bdce-f990e4fb7830 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.598519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd289da-5f8f-4340-bdce-f990e4fb7830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.600068 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd289da-5f8f-4340-bdce-f990e4fb7830 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.601365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd289da-5f8f-4340-bdce-f990e4fb7830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:04.604572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd289da-5f8f-4340-bdce-f990e4fb7830 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:04.654555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1347/2700] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:04 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:04.660558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd289da-5f8f-4340-bdce-f990e4fb7830 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:04.690943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8a72152aa74463a4d321827f18dd45', 'request_id': 'req-49fb105a-c56f-4767-ad56-afcd0529f861'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.701396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8a72152aa74463a4d321827f18dd45', 'request_id': 'req-49fb105a-c56f-4767-ad56-afcd0529f861'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.711835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8a72152aa74463a4d321827f18dd45', 'request_id': 'req-49fb105a-c56f-4767-ad56-afcd0529f861'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.731120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] The token cache is being invalidate because project 2e8a72152aa74463a4d321827f18dd45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:04.731385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.732388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.739246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1354/2701] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:04 2022] DELETE /identity/v3/projects/2e8a72152aa74463a4d321827f18dd45 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:04.740194 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:04.741649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:04.786130 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:04.834663 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd289da-5f8f-4340-bdce-f990e4fb7830 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:04.837285 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb105a-c56f-4767-ad56-afcd0529f861 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:04.844222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:04.847784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:04.848357 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.848694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.848805 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.848899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] PATH_INFO: `/v3/users/75ecce90109746978e2ec70b43263113` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.849070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.849396 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.850357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:04.851513 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ecce90109746978e2ec70b43263113)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:04.852986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:04.856660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:04.888535 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1348/2702] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:04 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:04.893830 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:04.896824 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:04.913715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b6764-4474-4cbb-b8ec-17c511d93079 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:04.919388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461d460-da16-478b-80ac-1b69eed88ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:04.920267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461d460-da16-478b-80ac-1b69eed88ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.920468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461d460-da16-478b-80ac-1b69eed88ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.920811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461d460-da16-478b-80ac-1b69eed88ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.921560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461d460-da16-478b-80ac-1b69eed88ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:04.924699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461d460-da16-478b-80ac-1b69eed88ca5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:04.936462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1349/2703] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:04.940353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461d460-da16-478b-80ac-1b69eed88ca5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:04.942030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ecce90109746978e2ec70b43263113'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.956600 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461d460-da16-478b-80ac-1b69eed88ca5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:04.963664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37657b-61ff-4a19-8676-f5ddea618212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:04.964745 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37657b-61ff-4a19-8676-f5ddea618212 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:04.965131 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37657b-61ff-4a19-8676-f5ddea618212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:04.965784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37657b-61ff-4a19-8676-f5ddea618212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:04.966916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37657b-61ff-4a19-8676-f5ddea618212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:04.969541 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ecce90109746978e2ec70b43263113'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.969691 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37657b-61ff-4a19-8676-f5ddea618212 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:04.972113 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ecce90109746978e2ec70b43263113'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:04.987791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1350/2704] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:04.989472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37657b-61ff-4a19-8676-f5ddea618212 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:04.992199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ecce90109746978e2ec70b43263113'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:05.004580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37657b-61ff-4a19-8676-f5ddea618212 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:05.009619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1386bc9c-1c36-43dc-8a97-6080f52a9d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ecce90109746978e2ec70b43263113'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:05.012372 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:05.014520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:05.015526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:05.018656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] PATH_INFO: `/v3/projects/860d3a4643d942cab542479ec5cb5d1e/users/b73238d6ce6940dd932a9bb6047d8d0b/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:05.038625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:05.046350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1355/2705] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:04 2022] DELETE /identity/v3/users/75ecce90109746978e2ec70b43263113 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:05.052137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] RBAC: Authorizing `identity:create_grant(project_id=860d3a4643d942cab542479ec5cb5d1e, user_id=b73238d6ce6940dd932a9bb6047d8d0b, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:05.054486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:05.075439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1351/2706] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:04 2022] PUT /identity/v3/projects/860d3a4643d942cab542479ec5cb5d1e/users/b73238d6ce6940dd932a9bb6047d8d0b/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:05.085740 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06316-4363-4233-8c29-573e45a844f2 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:05.086388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06316-4363-4233-8c29-573e45a844f2 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:05.086883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d06316-4363-4233-8c29-573e45a844f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:05.137468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-56d06316-4363-4233-8c29-573e45a844f2 None None] MFA Rules not processed for user `b73238d6ce6940dd932a9bb6047d8d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:05.144040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-56d06316-4363-4233-8c29-573e45a844f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:05.289214 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1356/2707] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:05.294277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:05.296471 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:05.476819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbd66d-1d87-468d-bdbe-56e3faaa388e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:05.482476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e771a0-9e7b-41bf-a55b-68ceeddf8b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:05.483017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e771a0-9e7b-41bf-a55b-68ceeddf8b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:05.483444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e771a0-9e7b-41bf-a55b-68ceeddf8b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:05.496772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e771a0-9e7b-41bf-a55b-68ceeddf8b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:05.497715 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e771a0-9e7b-41bf-a55b-68ceeddf8b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:05.501017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e771a0-9e7b-41bf-a55b-68ceeddf8b39 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:05.561558 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06316-4363-4233-8c29-573e45a844f2 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:05.562970 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-56d06316-4363-4233-8c29-573e45a844f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:05.567909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1352/2708] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:05 2022] POST /identity/v3/projects => generated 377 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:05.572196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e771a0-9e7b-41bf-a55b-68ceeddf8b39 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:05.609463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d06316-4363-4233-8c29-573e45a844f2 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:05.719196 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:05.766489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e771a0-9e7b-41bf-a55b-68ceeddf8b39 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:05.783553 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83453b12-defd-4428-a221-53557450f9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:05.784380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453b12-defd-4428-a221-53557450f9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:05.785646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453b12-defd-4428-a221-53557450f9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:05.786208 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453b12-defd-4428-a221-53557450f9d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:05.786856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83453b12-defd-4428-a221-53557450f9d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:05.788284 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:05.789385 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:05.790763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:05.791307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] PATH_INFO: `/v3/projects/aee1921ffa4848b7922b2e9fb26102c5` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:05.792499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83453b12-defd-4428-a221-53557450f9d3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:05.793909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee1921ffa4848b7922b2e9fb26102c5)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:05.805790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:05.841844 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1353/2709] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:05 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:05.845257 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83453b12-defd-4428-a221-53557450f9d3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:05.868584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83453b12-defd-4428-a221-53557450f9d3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:05.874536 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8fdf7-86bb-4220-82de-104750addf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:05.875174 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8fdf7-86bb-4220-82de-104750addf0a admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:05.875455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8fdf7-86bb-4220-82de-104750addf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:05.875800 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8fdf7-86bb-4220-82de-104750addf0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:05.877232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8fdf7-86bb-4220-82de-104750addf0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:05.878572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8fdf7-86bb-4220-82de-104750addf0a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:05.891339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1354/2710] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:05.896991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8fdf7-86bb-4220-82de-104750addf0a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:05.919618 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8fdf7-86bb-4220-82de-104750addf0a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:05.933085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f590cd1-2420-44c2-bb13-0c5820ecf3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:05.933962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f590cd1-2420-44c2-bb13-0c5820ecf3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:05.935517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f590cd1-2420-44c2-bb13-0c5820ecf3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:05.937476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f590cd1-2420-44c2-bb13-0c5820ecf3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:05.938151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f590cd1-2420-44c2-bb13-0c5820ecf3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:05.946193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f590cd1-2420-44c2-bb13-0c5820ecf3a0 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:05.969027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1355/2711] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:05.973029 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f590cd1-2420-44c2-bb13-0c5820ecf3a0 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:05.990205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f590cd1-2420-44c2-bb13-0c5820ecf3a0 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:05.990883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee1921ffa4848b7922b2e9fb26102c5', 'request_id': 'req-1a094036-58e4-466a-ae3e-9f5999c674dd'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:06.026973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee1921ffa4848b7922b2e9fb26102c5', 'request_id': 'req-1a094036-58e4-466a-ae3e-9f5999c674dd'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:06.061935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:06.074431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee1921ffa4848b7922b2e9fb26102c5', 'request_id': 'req-1a094036-58e4-466a-ae3e-9f5999c674dd'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:06.108042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbec006-058a-49eb-9d61-13047f562346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.108889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec006-058a-49eb-9d61-13047f562346 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.109059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec006-058a-49eb-9d61-13047f562346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.110100 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec006-058a-49eb-9d61-13047f562346 admin admin] PATH_INFO: `/v3/projects/1bd852dc9350427ea7c5634932a02965/users/d421c4d8b73b49d3906bb30fd033e851/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.128484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] The token cache is being invalidate because project aee1921ffa4848b7922b2e9fb26102c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:06.129837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:06.132752 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-1a094036-58e4-466a-ae3e-9f5999c674dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:06.137067 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1357/2712] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:05 2022] DELETE /identity/v3/projects/aee1921ffa4848b7922b2e9fb26102c5 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:06.143459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25c233-3263-4595-bca8-d8b4321fee1d None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.144806 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25c233-3263-4595-bca8-d8b4321fee1d None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.145580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25c233-3263-4595-bca8-d8b4321fee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.149811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbec006-058a-49eb-9d61-13047f562346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd852dc9350427ea7c5634932a02965, user_id=d421c4d8b73b49d3906bb30fd033e851, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.153151 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbec006-058a-49eb-9d61-13047f562346 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.184349 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1356/2713] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:05 2022] PUT /identity/v3/projects/1bd852dc9350427ea7c5634932a02965/users/d421c4d8b73b49d3906bb30fd033e851/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:06.194467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9d6aa-736a-474d-a2f4-07151bc997c2 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.195070 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9d6aa-736a-474d-a2f4-07151bc997c2 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.195546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9d6aa-736a-474d-a2f4-07151bc997c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.206761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4d25c233-3263-4595-bca8-d8b4321fee1d None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:06.208117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4d25c233-3263-4595-bca8-d8b4321fee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:06.247252 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-79d9d6aa-736a-474d-a2f4-07151bc997c2 None None] MFA Rules not processed for user `d421c4d8b73b49d3906bb30fd033e851`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:06.249235 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-79d9d6aa-736a-474d-a2f4-07151bc997c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:06.283744 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1358/2714] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:06.293111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0cd94-f343-49ff-8c49-38f75cc2143b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.293454 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0cd94-f343-49ff-8c49-38f75cc2143b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.294250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0cd94-f343-49ff-8c49-38f75cc2143b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.319548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-77d0cd94-f343-49ff-8c49-38f75cc2143b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:06.320719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-77d0cd94-f343-49ff-8c49-38f75cc2143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:06.332087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1359/2715] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:06.349825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0cd94-f343-49ff-8c49-38f75cc2143b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.367534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0cd94-f343-49ff-8c49-38f75cc2143b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.376202 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1357/2716] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:06.381622 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9d6aa-736a-474d-a2f4-07151bc997c2 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.383626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-79d9d6aa-736a-474d-a2f4-07151bc997c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:06.385937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.388236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.388536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.390001 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.390774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:06.391085 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.397502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.406695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9d6aa-736a-474d-a2f4-07151bc997c2 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.413932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab16d5-bbfe-49c2-81a4-cef51e14fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.414891 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab16d5-bbfe-49c2-81a4-cef51e14fe13 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.415240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab16d5-bbfe-49c2-81a4-cef51e14fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.415730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab16d5-bbfe-49c2-81a4-cef51e14fe13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.417799 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab16d5-bbfe-49c2-81a4-cef51e14fe13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.421949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab16d5-bbfe-49c2-81a4-cef51e14fe13 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.426365 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1360/2717] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 17:04:06.430729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.449641 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82e0f4-1c49-4cd0-aa3f-4bf7f5a36670 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.455954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc5df4-eb3d-48f2-a5a3-c4bbd6ed02a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.456614 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5df4-eb3d-48f2-a5a3-c4bbd6ed02a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.456881 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5df4-eb3d-48f2-a5a3-c4bbd6ed02a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.457151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc5df4-eb3d-48f2-a5a3-c4bbd6ed02a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.457778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc5df4-eb3d-48f2-a5a3-c4bbd6ed02a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.459704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc5df4-eb3d-48f2-a5a3-c4bbd6ed02a3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.498187 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1358/2718] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:06 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:06.498767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1361/2719] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:06 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:06.500215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab16d5-bbfe-49c2-81a4-cef51e14fe13 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.504035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc5df4-eb3d-48f2-a5a3-c4bbd6ed02a3 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.547835 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:06.590528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc5df4-eb3d-48f2-a5a3-c4bbd6ed02a3 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.596249 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab16d5-bbfe-49c2-81a4-cef51e14fe13 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.596771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec605-d5e3-43dd-9724-3a99f51c997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.597389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec605-d5e3-43dd-9724-3a99f51c997a admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.597720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec605-d5e3-43dd-9724-3a99f51c997a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.597912 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ec605-d5e3-43dd-9724-3a99f51c997a admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.598679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ec605-d5e3-43dd-9724-3a99f51c997a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.600763 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ec605-d5e3-43dd-9724-3a99f51c997a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.615692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a78987-d06d-46e1-ba5b-a4d5e8b5b158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.616537 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a78987-d06d-46e1-ba5b-a4d5e8b5b158 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.616963 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a78987-d06d-46e1-ba5b-a4d5e8b5b158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.617356 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a78987-d06d-46e1-ba5b-a4d5e8b5b158 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.618354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a78987-d06d-46e1-ba5b-a4d5e8b5b158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.620696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a78987-d06d-46e1-ba5b-a4d5e8b5b158 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.635025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1362/2720] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:06 2022] POST /identity/v3/users => generated 285 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:06.639775 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec605-d5e3-43dd-9724-3a99f51c997a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.651007 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ec605-d5e3-43dd-9724-3a99f51c997a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.657432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82c42-7d44-4fb3-8843-ac13e62fb3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.658375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82c42-7d44-4fb3-8843-ac13e62fb3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.658830 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82c42-7d44-4fb3-8843-ac13e62fb3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.659266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82c42-7d44-4fb3-8843-ac13e62fb3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.660026 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1359/2721] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:06 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:06.660447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b82c42-7d44-4fb3-8843-ac13e62fb3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.662216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b82c42-7d44-4fb3-8843-ac13e62fb3f9 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.665431 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a78987-d06d-46e1-ba5b-a4d5e8b5b158 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.674788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1363/2722] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:06.679592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82c42-7d44-4fb3-8843-ac13e62fb3f9 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.682907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a78987-d06d-46e1-ba5b-a4d5e8b5b158 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.692120 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82c42-7d44-4fb3-8843-ac13e62fb3f9 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.692360 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e75b77-f4b4-4f1b-8c65-197805fcdfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.692880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75b77-f4b4-4f1b-8c65-197805fcdfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.693231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75b77-f4b4-4f1b-8c65-197805fcdfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.693513 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e75b77-f4b4-4f1b-8c65-197805fcdfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.695470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e75b77-f4b4-4f1b-8c65-197805fcdfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.698998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e75b77-f4b4-4f1b-8c65-197805fcdfe3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.699863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cec3c-8cd3-4423-a70b-586572056462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.700963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cec3c-8cd3-4423-a70b-586572056462 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.701417 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cec3c-8cd3-4423-a70b-586572056462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.702087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cec3c-8cd3-4423-a70b-586572056462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.702900 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cec3c-8cd3-4423-a70b-586572056462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.705872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cec3c-8cd3-4423-a70b-586572056462 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.718808 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1364/2723] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:06.724490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cec3c-8cd3-4423-a70b-586572056462 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.726893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1360/2724] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:06.730648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e75b77-f4b4-4f1b-8c65-197805fcdfe3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.738915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cec3c-8cd3-4423-a70b-586572056462 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.745685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86f032-2450-4343-a24e-319ccf4806ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.746563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86f032-2450-4343-a24e-319ccf4806ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.747026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86f032-2450-4343-a24e-319ccf4806ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.747502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c86f032-2450-4343-a24e-319ccf4806ee admin admin] PATH_INFO: `/v3/projects/decc7c2316f64b72b14be45faa38e5c1/users/20f3b7fe0dcc4443b5528f9aabc04095/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.756425 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e75b77-f4b4-4f1b-8c65-197805fcdfe3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.770973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601b92e-e2a5-4ab2-8b4e-2a003559a069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.771670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601b92e-e2a5-4ab2-8b4e-2a003559a069 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.771951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601b92e-e2a5-4ab2-8b4e-2a003559a069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.772266 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601b92e-e2a5-4ab2-8b4e-2a003559a069 admin admin] PATH_INFO: `/v3/projects/570143e2306d43fca062f7c0e2d46acf/users/253a540ab8f0477eb9ac7414a7644c4d/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.773313 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86f032-2450-4343-a24e-319ccf4806ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=decc7c2316f64b72b14be45faa38e5c1, user_id=20f3b7fe0dcc4443b5528f9aabc04095, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.780483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c86f032-2450-4343-a24e-319ccf4806ee admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.792903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:06.815071 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1365/2725] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:06 2022] PUT /identity/v3/projects/decc7c2316f64b72b14be45faa38e5c1/users/20f3b7fe0dcc4443b5528f9aabc04095/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:06.815690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601b92e-e2a5-4ab2-8b4e-2a003559a069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=570143e2306d43fca062f7c0e2d46acf, user_id=253a540ab8f0477eb9ac7414a7644c4d, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.819165 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601b92e-e2a5-4ab2-8b4e-2a003559a069 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:06.840084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1361/2726] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:06 2022] PUT /identity/v3/projects/570143e2306d43fca062f7c0e2d46acf/users/253a540ab8f0477eb9ac7414a7644c4d/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:06.844746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86f032-2450-4343-a24e-319ccf4806ee admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:06.890439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f288684-eeb3-449b-878c-bea8454a7aec None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.890819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f288684-eeb3-449b-878c-bea8454a7aec None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.891017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f288684-eeb3-449b-878c-bea8454a7aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.921898 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c86f032-2450-4343-a24e-319ccf4806ee admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:06.952215 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-6f288684-eeb3-449b-878c-bea8454a7aec None None] MFA Rules not processed for user `20f3b7fe0dcc4443b5528f9aabc04095`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:06.953626 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6f288684-eeb3-449b-878c-bea8454a7aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:06.989990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce5a11-d257-4102-b230-bfc04a2a6a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:06.990611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce5a11-d257-4102-b230-bfc04a2a6a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:06.990889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce5a11-d257-4102-b230-bfc04a2a6a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:06.991319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce5a11-d257-4102-b230-bfc04a2a6a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:06.992119 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce5a11-d257-4102-b230-bfc04a2a6a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:06.993743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce5a11-d257-4102-b230-bfc04a2a6a43 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.008040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1366/2727] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:07.008964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce5a11-d257-4102-b230-bfc04a2a6a43 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.019264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce5a11-d257-4102-b230-bfc04a2a6a43 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:07.024494 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d5dd3-0f0c-4d0e-b1fb-20f7ea6667e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:07.025162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d5dd3-0f0c-4d0e-b1fb-20f7ea6667e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.025439 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d5dd3-0f0c-4d0e-b1fb-20f7ea6667e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.025734 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d5dd3-0f0c-4d0e-b1fb-20f7ea6667e3 admin admin] PATH_INFO: `/v3/projects/570143e2306d43fca062f7c0e2d46acf/users/253a540ab8f0477eb9ac7414a7644c4d/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.027755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d5dd3-0f0c-4d0e-b1fb-20f7ea6667e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=570143e2306d43fca062f7c0e2d46acf, user_id=253a540ab8f0477eb9ac7414a7644c4d, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:07.030502 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d5dd3-0f0c-4d0e-b1fb-20f7ea6667e3 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.038115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:07.043066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1367/2728] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:07 2022] PUT /identity/v3/projects/570143e2306d43fca062f7c0e2d46acf/users/253a540ab8f0477eb9ac7414a7644c4d/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:07.058371 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a367-5fee-48ed-94b9-9a46e1c9ec05 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.058647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a367-5fee-48ed-94b9-9a46e1c9ec05 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.058923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8a367-5fee-48ed-94b9-9a46e1c9ec05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.069691 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1362/2729] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:07.074742 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f288684-eeb3-449b-878c-bea8454a7aec None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.092236 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f288684-eeb3-449b-878c-bea8454a7aec None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:07.098382 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-64f8a367-5fee-48ed-94b9-9a46e1c9ec05 None None] MFA Rules not processed for user `253a540ab8f0477eb9ac7414a7644c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:07.099611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-64f8a367-5fee-48ed-94b9-9a46e1c9ec05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:07.187635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1368/2730] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:07.199647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230a138-e429-4f9b-8719-5ff99fdb6ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:07.200314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a138-e429-4f9b-8719-5ff99fdb6ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.200593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a138-e429-4f9b-8719-5ff99fdb6ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.201125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a138-e429-4f9b-8719-5ff99fdb6ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.201768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230a138-e429-4f9b-8719-5ff99fdb6ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:07.203831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230a138-e429-4f9b-8719-5ff99fdb6ec9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.228992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8a367-5fee-48ed-94b9-9a46e1c9ec05 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.230644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-64f8a367-5fee-48ed-94b9-9a46e1c9ec05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:07.259951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1363/2731] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:07 2022] POST /identity/v3/projects => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:07.271483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230a138-e429-4f9b-8719-5ff99fdb6ec9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.343077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8a367-5fee-48ed-94b9-9a46e1c9ec05 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:07.357851 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230a138-e429-4f9b-8719-5ff99fdb6ec9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:07.363624 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7540037-d741-424d-b495-cef7aa04b94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:07.364753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7540037-d741-424d-b495-cef7aa04b94f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.365130 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7540037-d741-424d-b495-cef7aa04b94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.365468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7540037-d741-424d-b495-cef7aa04b94f admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.366334 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7540037-d741-424d-b495-cef7aa04b94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:07.368605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7540037-d741-424d-b495-cef7aa04b94f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.378748 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:07.403778 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1364/2732] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:07 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:07.406321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:07.408170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7540037-d741-424d-b495-cef7aa04b94f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.410188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.410488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.412169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.416531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:07.431543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7540037-d741-424d-b495-cef7aa04b94f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:07.437437 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1070b-ec21-4fd7-a42c-c5e97e6b3dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:07.438123 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1070b-ec21-4fd7-a42c-c5e97e6b3dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.438408 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1070b-ec21-4fd7-a42c-c5e97e6b3dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.438683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1070b-ec21-4fd7-a42c-c5e97e6b3dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.440489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1070b-ec21-4fd7-a42c-c5e97e6b3dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:07.441822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1070b-ec21-4fd7-a42c-c5e97e6b3dc3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.453125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1365/2733] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:07.457231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1070b-ec21-4fd7-a42c-c5e97e6b3dc3 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.473398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1070b-ec21-4fd7-a42c-c5e97e6b3dc3 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:07.482464 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d585f4-a9c3-475a-8cae-4346d531f3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:07.483112 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d585f4-a9c3-475a-8cae-4346d531f3fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.483395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d585f4-a9c3-475a-8cae-4346d531f3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.483666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d585f4-a9c3-475a-8cae-4346d531f3fa admin admin] PATH_INFO: `/v3/projects/5e3ae87b813d48e6a0cb8b77c3176afe/users/e782a6f97d58446399962e9232190946/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.500856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d585f4-a9c3-475a-8cae-4346d531f3fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3ae87b813d48e6a0cb8b77c3176afe, user_id=e782a6f97d58446399962e9232190946, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:07.504141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d585f4-a9c3-475a-8cae-4346d531f3fa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.538406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1366/2734] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:07 2022] PUT /identity/v3/projects/5e3ae87b813d48e6a0cb8b77c3176afe/users/e782a6f97d58446399962e9232190946/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:07.538406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d585f4-a9c3-475a-8cae-4346d531f3fa admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.542536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:07.545289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:07.547114 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.571022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1369/2735] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:07.651021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d585f4-a9c3-475a-8cae-4346d531f3fa admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:07.655880 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520d348-2b95-44f2-8f9d-5681bd276ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:07.656526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520d348-2b95-44f2-8f9d-5681bd276ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.656827 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520d348-2b95-44f2-8f9d-5681bd276ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.657210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520d348-2b95-44f2-8f9d-5681bd276ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.658293 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520d348-2b95-44f2-8f9d-5681bd276ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:07.660604 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520d348-2b95-44f2-8f9d-5681bd276ac1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.676482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1367/2736] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:07.677758 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.694689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400a221-05b6-478d-8eec-51e6ad3ac5c4 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:07.711617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:07.714077 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.714362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.716329 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] PATH_INFO: `/v3/projects/5e3ae87b813d48e6a0cb8b77c3176afe/users/e782a6f97d58446399962e9232190946/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.718923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3ae87b813d48e6a0cb8b77c3176afe, user_id=e782a6f97d58446399962e9232190946, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:07.722798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:07.746977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e9e2-f915-4477-baee-86541fcf63b4 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:07.747289 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e9e2-f915-4477-baee-86541fcf63b4 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:07.747531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e9e2-f915-4477-baee-86541fcf63b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:07.750197 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1370/2737] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:07 2022] PUT /identity/v3/projects/5e3ae87b813d48e6a0cb8b77c3176afe/users/e782a6f97d58446399962e9232190946/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:07.799926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-7805e9e2-f915-4477-baee-86541fcf63b4 None None] MFA Rules not processed for user `e782a6f97d58446399962e9232190946`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:07.801214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7805e9e2-f915-4477-baee-86541fcf63b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:07.891467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1368/2738] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:07.904879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:07.916440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:08.022479 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3255351-6562-410d-8e9d-a41f5c82aa7c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:08.026341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae24066-6dad-4ce4-8994-795b41622ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:08.027131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24066-6dad-4ce4-8994-795b41622ffe service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:08.027524 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24066-6dad-4ce4-8994-795b41622ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:08.027928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24066-6dad-4ce4-8994-795b41622ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:08.038490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae24066-6dad-4ce4-8994-795b41622ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:08.043794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae24066-6dad-4ce4-8994-795b41622ffe service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:08.050851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1371/2739] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:09.815141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e9e2-f915-4477-baee-86541fcf63b4 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:09.874112 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:09.899160 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805e9e2-f915-4477-baee-86541fcf63b4 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:09.904296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:09.905074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:09.905357 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:09.905623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] PATH_INFO: `/v3/users/20f3b7fe0dcc4443b5528f9aabc04095` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:09.906750 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f3b7fe0dcc4443b5528f9aabc04095)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:09.909406 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:09.959153 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f3b7fe0dcc4443b5528f9aabc04095'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:09.966961 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f3b7fe0dcc4443b5528f9aabc04095'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:09.967708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f3b7fe0dcc4443b5528f9aabc04095'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:09.982292 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f3b7fe0dcc4443b5528f9aabc04095'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:09.983005 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f3b7fe0dcc4443b5528f9aabc04095'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:09.998519 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1369/2740] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:09 2022] DELETE /identity/v3/users/20f3b7fe0dcc4443b5528f9aabc04095 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:10.003169 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd0e9-9854-43ae-b691-aef83e055104 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:10.003730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd0e9-9854-43ae-b691-aef83e055104 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:10.003730 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd0e9-9854-43ae-b691-aef83e055104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:10.025111 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-0b3dd0e9-9854-43ae-b691-aef83e055104 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:10.026598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0b3dd0e9-9854-43ae-b691-aef83e055104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:10.172812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1372/2741] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:10.185316 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:10.187098 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:10.313691 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e550add-9d24-4708-8f74-917a7563dd10 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:10.324497 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:10.325271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:10.325660 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:10.326294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:10.341436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:10.345951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:10.359332 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1370/2742] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:10.381581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd0e9-9854-43ae-b691-aef83e055104 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:10.382960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0b3dd0e9-9854-43ae-b691-aef83e055104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:10.461485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd0e9-9854-43ae-b691-aef83e055104 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:10.467528 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:10.468281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:10.469372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:10.469668 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:10.472131 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:10.539022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:10.557212 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:10.558631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:10.571255 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1373/2743] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:10.710520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:10.712549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:10.729327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307f05f-a357-4f72-b29d-84ddef58ea96 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:10.734941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:10.735369 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:10.736564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:10.736793 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] PATH_INFO: `/v3/projects/decc7c2316f64b72b14be45faa38e5c1` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:10.738164 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=decc7c2316f64b72b14be45faa38e5c1)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:10.740336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:10.807307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'decc7c2316f64b72b14be45faa38e5c1', 'request_id': 'req-ea3e2b5a-028a-4e21-a189-0cda29512e11'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:10.814683 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:10.832192 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'decc7c2316f64b72b14be45faa38e5c1', 'request_id': 'req-ea3e2b5a-028a-4e21-a189-0cda29512e11'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:10.849189 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'decc7c2316f64b72b14be45faa38e5c1', 'request_id': 'req-ea3e2b5a-028a-4e21-a189-0cda29512e11'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:10.872242 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] The token cache is being invalidate because project decc7c2316f64b72b14be45faa38e5c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:10.872633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:10.873787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:10.877690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1371/2744] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:10 2022] DELETE /identity/v3/projects/decc7c2316f64b72b14be45faa38e5c1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:10.881991 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:10.893430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbf69d-779e-4399-872a-76ab3ea926b8 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:10.956143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:10.956871 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:10.957209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:10.957481 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] PATH_INFO: `/v3/users/e782a6f97d58446399962e9232190946` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:10.958622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e782a6f97d58446399962e9232190946)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:10.960576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.012347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e782a6f97d58446399962e9232190946'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.029609 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e782a6f97d58446399962e9232190946'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.030725 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e782a6f97d58446399962e9232190946'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.039395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e782a6f97d58446399962e9232190946'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.049915 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e782a6f97d58446399962e9232190946'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.066670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1374/2745] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:10 2022] DELETE /identity/v3/users/e782a6f97d58446399962e9232190946 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:11.259348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:11.260796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:11.319098 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e2b5a-028a-4e21-a189-0cda29512e11 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:11.359189 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:11.361459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:11.373528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37440ff3-3933-40ba-8b70-21055022ddf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:11.374221 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37440ff3-3933-40ba-8b70-21055022ddf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.374503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37440ff3-3933-40ba-8b70-21055022ddf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.374773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37440ff3-3933-40ba-8b70-21055022ddf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.377529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-37440ff3-3933-40ba-8b70-21055022ddf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:11.430580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f369eb-ad77-4d88-b141-669dfe0dcdfc admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:11.435134 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:11.435755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.436031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.436333 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] PATH_INFO: `/v3/projects/5e3ae87b813d48e6a0cb8b77c3176afe` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.437435 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3ae87b813d48e6a0cb8b77c3176afe)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:11.439591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.441081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:11.445355 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37440ff3-3933-40ba-8b70-21055022ddf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:11.447268 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37440ff3-3933-40ba-8b70-21055022ddf5 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.458872 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1372/2746] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:11.490460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3ae87b813d48e6a0cb8b77c3176afe', 'request_id': 'req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.496844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3ae87b813d48e6a0cb8b77c3176afe', 'request_id': 'req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.502984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3ae87b813d48e6a0cb8b77c3176afe', 'request_id': 'req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.517959 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] The token cache is being invalidate because project 5e3ae87b813d48e6a0cb8b77c3176afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:11.518253 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.518923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-83d8f74c-2fe0-4d25-a256-2580c2f3ddff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:11.520741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1375/2747] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:11 2022] DELETE /identity/v3/projects/5e3ae87b813d48e6a0cb8b77c3176afe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:11.525926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e7a6-a468-4496-b86a-28ba952dabf5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.526378 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e7a6-a468-4496-b86a-28ba952dabf5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.526751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e7a6-a468-4496-b86a-28ba952dabf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.550219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-52b8e7a6-a468-4496-b86a-28ba952dabf5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:11.551439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-52b8e7a6-a468-4496-b86a-28ba952dabf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:11.602526 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:11.619943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1373/2748] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:11.629570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfef126-e978-4910-803c-3eee00dcaa02 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.629935 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfef126-e978-4910-803c-3eee00dcaa02 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.630275 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfef126-e978-4910-803c-3eee00dcaa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.653861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5bfef126-e978-4910-803c-3eee00dcaa02 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:11.655065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5bfef126-e978-4910-803c-3eee00dcaa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:11.668807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1376/2749] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:11.670661 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e7a6-a468-4496-b86a-28ba952dabf5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:11.681177 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e7a6-a468-4496-b86a-28ba952dabf5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:11.686329 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:11.687015 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.687365 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.687708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.688458 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:11.689441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:11.691474 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.706477 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1374/2750] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:04:11.712047 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfef126-e978-4910-803c-3eee00dcaa02 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:11.723237 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfef126-e978-4910-803c-3eee00dcaa02 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:11.728771 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e9667-e8ee-4871-b34e-348a8cb3a5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:11.729523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9667-e8ee-4871-b34e-348a8cb3a5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.729772 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9667-e8ee-4871-b34e-348a8cb3a5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.730060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e9667-e8ee-4871-b34e-348a8cb3a5f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.730690 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e9667-e8ee-4871-b34e-348a8cb3a5f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:11.732591 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e9667-e8ee-4871-b34e-348a8cb3a5f7 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.760776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1377/2751] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:11 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:11.765008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:11.775873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98e655-fede-47fa-b7df-047b901e2428 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:11.834231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2305d-8292-4fc6-b1d7-59e8a56d6d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:11.834970 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2305d-8292-4fc6-b1d7-59e8a56d6d92 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.835152 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2305d-8292-4fc6-b1d7-59e8a56d6d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.835424 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2305d-8292-4fc6-b1d7-59e8a56d6d92 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.836109 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2305d-8292-4fc6-b1d7-59e8a56d6d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:11.838340 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2305d-8292-4fc6-b1d7-59e8a56d6d92 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.860448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1375/2752] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:11 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:11.865136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e9667-e8ee-4871-b34e-348a8cb3a5f7 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:11.879336 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e9667-e8ee-4871-b34e-348a8cb3a5f7 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:11.884182 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba1a03-a371-40c4-be36-2c63d0ff0684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:11.884799 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba1a03-a371-40c4-be36-2c63d0ff0684 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.885168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba1a03-a371-40c4-be36-2c63d0ff0684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.885509 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba1a03-a371-40c4-be36-2c63d0ff0684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.886227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba1a03-a371-40c4-be36-2c63d0ff0684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:11.888058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba1a03-a371-40c4-be36-2c63d0ff0684 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.899475 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1378/2753] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:11.903828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2305d-8292-4fc6-b1d7-59e8a56d6d92 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:11.915065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2305d-8292-4fc6-b1d7-59e8a56d6d92 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:11.919761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f384c-7119-4d4a-adeb-57317bfecee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:11.920393 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f384c-7119-4d4a-adeb-57317bfecee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.920671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f384c-7119-4d4a-adeb-57317bfecee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.920942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f384c-7119-4d4a-adeb-57317bfecee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.921586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f384c-7119-4d4a-adeb-57317bfecee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:11.923233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f384c-7119-4d4a-adeb-57317bfecee3 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.933726 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1376/2754] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:11.938033 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba1a03-a371-40c4-be36-2c63d0ff0684 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:11.948076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba1a03-a371-40c4-be36-2c63d0ff0684 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:11.953062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1376ab5-55a6-4aa2-9b47-ef728187f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:11.953710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376ab5-55a6-4aa2-9b47-ef728187f528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.953992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376ab5-55a6-4aa2-9b47-ef728187f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.954265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376ab5-55a6-4aa2-9b47-ef728187f528 admin admin] PATH_INFO: `/v3/projects/04b1dc0422554ef89a475d6fa37f53f2/users/078aa3a96563461696e50ad53343aaef/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:11.969300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1376ab5-55a6-4aa2-9b47-ef728187f528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b1dc0422554ef89a475d6fa37f53f2, user_id=078aa3a96563461696e50ad53343aaef, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:11.972031 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1376ab5-55a6-4aa2-9b47-ef728187f528 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:11.987624 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1379/2755] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:11 2022] PUT /identity/v3/projects/04b1dc0422554ef89a475d6fa37f53f2/users/078aa3a96563461696e50ad53343aaef/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:11.997791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c27e-49d6-4fb2-a8a3-2e46268d4218 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:11.998159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c27e-49d6-4fb2-a8a3-2e46268d4218 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:11.998394 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c27e-49d6-4fb2-a8a3-2e46268d4218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.029753 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-c256c27e-49d6-4fb2-a8a3-2e46268d4218 None None] MFA Rules not processed for user `078aa3a96563461696e50ad53343aaef`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:12.031002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c256c27e-49d6-4fb2-a8a3-2e46268d4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:12.107817 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1377/2756] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:11 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:12.112566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1376ab5-55a6-4aa2-9b47-ef728187f528 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:12.175025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1376ab5-55a6-4aa2-9b47-ef728187f528 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:12.179937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db549f9a-b664-49f9-9b83-cd5896e8009e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:12.180571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db549f9a-b664-49f9-9b83-cd5896e8009e admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.180850 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db549f9a-b664-49f9-9b83-cd5896e8009e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.181117 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db549f9a-b664-49f9-9b83-cd5896e8009e admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.181739 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db549f9a-b664-49f9-9b83-cd5896e8009e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:12.183628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db549f9a-b664-49f9-9b83-cd5896e8009e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:12.210856 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1380/2757] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:12 2022] POST /identity/v3/projects => generated 399 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:12.214896 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256c27e-49d6-4fb2-a8a3-2e46268d4218 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:12.225625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256c27e-49d6-4fb2-a8a3-2e46268d4218 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:12.266310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:12.282940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0c7c8-c6f6-456f-a0c4-5a70087cee2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:12.283565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c7c8-c6f6-456f-a0c4-5a70087cee2c admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.283839 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c7c8-c6f6-456f-a0c4-5a70087cee2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.284125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c7c8-c6f6-456f-a0c4-5a70087cee2c admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.284754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0c7c8-c6f6-456f-a0c4-5a70087cee2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:12.286647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0c7c8-c6f6-456f-a0c4-5a70087cee2c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:12.306367 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1378/2758] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:12 2022] POST /identity/v3/users => generated 306 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:12.310444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db549f9a-b664-49f9-9b83-cd5896e8009e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:12.321343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db549f9a-b664-49f9-9b83-cd5896e8009e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:12.326179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3929-8bd5-4122-8937-3f190c63255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:12.326764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3929-8bd5-4122-8937-3f190c63255e admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.327042 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3929-8bd5-4122-8937-3f190c63255e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.327310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3929-8bd5-4122-8937-3f190c63255e admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.327951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf3929-8bd5-4122-8937-3f190c63255e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:12.330138 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf3929-8bd5-4122-8937-3f190c63255e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:12.335364 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:12.341749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1381/2759] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:12.346080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0c7c8-c6f6-456f-a0c4-5a70087cee2c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:12.356036 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0c7c8-c6f6-456f-a0c4-5a70087cee2c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:12.361311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bfaa5-ab94-4c23-a0e7-647420b56dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:12.361712 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bfaa5-ab94-4c23-a0e7-647420b56dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.361992 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bfaa5-ab94-4c23-a0e7-647420b56dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.362265 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818bfaa5-ab94-4c23-a0e7-647420b56dcb admin admin] PATH_INFO: `/v3/projects/c2f5f052ee044105b95ae1a2a54d289e/users/3110579258b942ba9468179f48c92c55/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.377191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bfaa5-ab94-4c23-a0e7-647420b56dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f5f052ee044105b95ae1a2a54d289e, user_id=3110579258b942ba9468179f48c92c55, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:12.380018 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818bfaa5-ab94-4c23-a0e7-647420b56dcb admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:12.393250 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1379/2760] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:12 2022] PUT /identity/v3/projects/c2f5f052ee044105b95ae1a2a54d289e/users/3110579258b942ba9468179f48c92c55/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:12.397235 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3929-8bd5-4122-8937-3f190c63255e admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:12.406975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3929-8bd5-4122-8937-3f190c63255e admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:12.462563 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03017e9-6b6a-4d0b-bb11-73a3704fc2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:12.463159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03017e9-6b6a-4d0b-bb11-73a3704fc2de admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.463440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03017e9-6b6a-4d0b-bb11-73a3704fc2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.463712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03017e9-6b6a-4d0b-bb11-73a3704fc2de admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.464379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03017e9-6b6a-4d0b-bb11-73a3704fc2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:12.465951 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03017e9-6b6a-4d0b-bb11-73a3704fc2de admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:12.476142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1382/2761] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:12.480362 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bfaa5-ab94-4c23-a0e7-647420b56dcb admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:12.491141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818bfaa5-ab94-4c23-a0e7-647420b56dcb admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:12.495665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:12.496331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.496635 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.496905 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] PATH_INFO: `/v3/projects/c2f5f052ee044105b95ae1a2a54d289e/users/3110579258b942ba9468179f48c92c55/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.498821 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f5f052ee044105b95ae1a2a54d289e, user_id=3110579258b942ba9468179f48c92c55, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:12.501469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:12.511664 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1380/2762] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:12 2022] PUT /identity/v3/projects/c2f5f052ee044105b95ae1a2a54d289e/users/3110579258b942ba9468179f48c92c55/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:12.523381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b103-de8a-41af-ace5-bce7df52bbb9 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.523670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b103-de8a-41af-ace5-bce7df52bbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.523943 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b103-de8a-41af-ace5-bce7df52bbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.551488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c623b103-de8a-41af-ace5-bce7df52bbb9 None None] MFA Rules not processed for user `3110579258b942ba9468179f48c92c55`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:12.552729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c623b103-de8a-41af-ace5-bce7df52bbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:12.622998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1383/2763] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:12 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:12.633625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:12.635414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:12.697184 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8bd68-e42f-445e-927d-ff8409ccfc43 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:12.701231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:12.701841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.702116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.702386 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.712002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:12.713455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:12.724695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1381/2764] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:12 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:12.874727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623b103-de8a-41af-ace5-bce7df52bbb9 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:12.886885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623b103-de8a-41af-ace5-bce7df52bbb9 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:12.890766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a113d-41a2-445e-ad5f-fe89431d7da3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:12.891445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a113d-41a2-445e-ad5f-fe89431d7da3 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:12.891796 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a113d-41a2-445e-ad5f-fe89431d7da3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:12.892372 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a113d-41a2-445e-ad5f-fe89431d7da3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:12.957990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a113d-41a2-445e-ad5f-fe89431d7da3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:12.959416 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a113d-41a2-445e-ad5f-fe89431d7da3 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:12.967986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1384/2765] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:12 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:15.108264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:15.109659 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:15.175422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cbc6b-b155-4918-bc46-53b165bb31fb service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:15.179969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:15.180623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:15.180902 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:15.181175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] PATH_INFO: `/v3/users/7cba2c8197d340cc8fb9a47fbb7a5460` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:15.182294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cba2c8197d340cc8fb9a47fbb7a5460)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:15.184237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:15.226594 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:15.232385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cba2c8197d340cc8fb9a47fbb7a5460'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.240469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cba2c8197d340cc8fb9a47fbb7a5460'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.241455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cba2c8197d340cc8fb9a47fbb7a5460'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.254415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cba2c8197d340cc8fb9a47fbb7a5460'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.255052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cba2c8197d340cc8fb9a47fbb7a5460'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.267141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1382/2766] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:15 2022] DELETE /identity/v3/users/7cba2c8197d340cc8fb9a47fbb7a5460 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:15.271571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb1dbf-e357-44a4-97cf-158f710e1e34 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:15.272088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb1dbf-e357-44a4-97cf-158f710e1e34 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:15.272240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb1dbf-e357-44a4-97cf-158f710e1e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:15.291984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-33eb1dbf-e357-44a4-97cf-158f710e1e34 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:15.293257 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-33eb1dbf-e357-44a4-97cf-158f710e1e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:15.307629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=100 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:15.355012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1385/2767] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:15.362997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:15.364414 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:15.365962 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb1dbf-e357-44a4-97cf-158f710e1e34 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:15.367338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-33eb1dbf-e357-44a4-97cf-158f710e1e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:15.377485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5ac95-0b52-4345-a5f7-2d2043821f52 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:15.382172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:15.382794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:15.383074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:15.383345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] PATH_INFO: `/v3/users/b73238d6ce6940dd932a9bb6047d8d0b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:15.384509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b73238d6ce6940dd932a9bb6047d8d0b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:15.386417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:15.426238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb1dbf-e357-44a4-97cf-158f710e1e34 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:15.429990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd9396-1d7b-4f1b-837a-f0bca01d5170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:15.430597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd9396-1d7b-4f1b-837a-f0bca01d5170 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:15.431300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd9396-1d7b-4f1b-837a-f0bca01d5170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:15.431300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd9396-1d7b-4f1b-837a-f0bca01d5170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:15.440064 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b73238d6ce6940dd932a9bb6047d8d0b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.441540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd9396-1d7b-4f1b-837a-f0bca01d5170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:15.443035 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd9396-1d7b-4f1b-837a-f0bca01d5170 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:15.447602 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b73238d6ce6940dd932a9bb6047d8d0b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.448518 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b73238d6ce6940dd932a9bb6047d8d0b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.454440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1386/2768] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:04:15.463848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b73238d6ce6940dd932a9bb6047d8d0b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.464732 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b73238d6ce6940dd932a9bb6047d8d0b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.479145 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1383/2769] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:15 2022] DELETE /identity/v3/users/b73238d6ce6940dd932a9bb6047d8d0b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:15.483682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2450a5-fdb0-4e95-bdf9-ae8d22dc3e6b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:15.483992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2450a5-fdb0-4e95-bdf9-ae8d22dc3e6b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:15.484290 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2450a5-fdb0-4e95-bdf9-ae8d22dc3e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:15.505550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5a2450a5-fdb0-4e95-bdf9-ae8d22dc3e6b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:15.506776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5a2450a5-fdb0-4e95-bdf9-ae8d22dc3e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:15.572719 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1387/2770] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:15.591399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:15.593646 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:15.661300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313fca5-4e0e-4d14-a97b-a4499ea8264a admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:15.665885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f947ac-848e-45a1-9bab-bc85e1f32f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:15.666503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f947ac-848e-45a1-9bab-bc85e1f32f9a service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:15.666783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f947ac-848e-45a1-9bab-bc85e1f32f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:15.667059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f947ac-848e-45a1-9bab-bc85e1f32f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:15.677811 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f947ac-848e-45a1-9bab-bc85e1f32f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:15.679943 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f947ac-848e-45a1-9bab-bc85e1f32f9a service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:15.690913 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1384/2771] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:15.802102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2450a5-fdb0-4e95-bdf9-ae8d22dc3e6b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:15.804151 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5a2450a5-fdb0-4e95-bdf9-ae8d22dc3e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:15.817941 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2450a5-fdb0-4e95-bdf9-ae8d22dc3e6b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:15.823854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:15.824877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:15.825156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:15.825451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] PATH_INFO: `/v3/projects/6b5a2925403842509420d0693ecb9570` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:15.826846 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b5a2925403842509420d0693ecb9570)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:15.830167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:15.900146 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b5a2925403842509420d0693ecb9570', 'request_id': 'req-32f3fe17-9e89-4671-a8b0-8438f699ffdc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.908594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b5a2925403842509420d0693ecb9570', 'request_id': 'req-32f3fe17-9e89-4671-a8b0-8438f699ffdc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.913662 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:15.915999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b5a2925403842509420d0693ecb9570', 'request_id': 'req-32f3fe17-9e89-4671-a8b0-8438f699ffdc'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.932608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] The token cache is being invalidate because project 6b5a2925403842509420d0693ecb9570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:15.933129 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.933885 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-32f3fe17-9e89-4671-a8b0-8438f699ffdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:15.935564 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1388/2772] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:15 2022] DELETE /identity/v3/projects/6b5a2925403842509420d0693ecb9570 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:15.941149 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416d82a-e861-4a53-a482-e0db3cf7811a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:15.941441 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416d82a-e861-4a53-a482-e0db3cf7811a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:15.942490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416d82a-e861-4a53-a482-e0db3cf7811a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:15.966448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e416d82a-e861-4a53-a482-e0db3cf7811a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:15.967549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e416d82a-e861-4a53-a482-e0db3cf7811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.003440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:16.048534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1385/2773] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:16.056689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18055771-4e32-4dbf-85d2-9c5504efbd36 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.056937 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18055771-4e32-4dbf-85d2-9c5504efbd36 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.057259 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18055771-4e32-4dbf-85d2-9c5504efbd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.078134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416d82a-e861-4a53-a482-e0db3cf7811a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.080699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e416d82a-e861-4a53-a482-e0db3cf7811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.084442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-18055771-4e32-4dbf-85d2-9c5504efbd36 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:16.085708 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-18055771-4e32-4dbf-85d2-9c5504efbd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.093949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416d82a-e861-4a53-a482-e0db3cf7811a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.097107 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1389/2774] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:16.102396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18055771-4e32-4dbf-85d2-9c5504efbd36 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.103630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.104384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.104662 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.104932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] PATH_INFO: `/v3/projects/860d3a4643d942cab542479ec5cb5d1e` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.106430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=860d3a4643d942cab542479ec5cb5d1e)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.108724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.116984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18055771-4e32-4dbf-85d2-9c5504efbd36 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.123445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.124274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.124598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.124897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.125586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:16.125950 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.128278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.141386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1390/2775] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:04:16.146882 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.156727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329fa37-2448-4d9d-a502-9a3c55be4d70 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.164547 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860d3a4643d942cab542479ec5cb5d1e', 'request_id': 'req-c589deb1-75fe-468e-bf9c-f58c0094f7ea'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.172969 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860d3a4643d942cab542479ec5cb5d1e', 'request_id': 'req-c589deb1-75fe-468e-bf9c-f58c0094f7ea'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.183483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860d3a4643d942cab542479ec5cb5d1e', 'request_id': 'req-c589deb1-75fe-468e-bf9c-f58c0094f7ea'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.197505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] The token cache is being invalidate because project 860d3a4643d942cab542479ec5cb5d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:16.197879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.198714 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.200555 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1386/2776] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:16 2022] DELETE /identity/v3/projects/860d3a4643d942cab542479ec5cb5d1e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:16.205417 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.207121 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.221343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.221953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.222233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.222503 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.223126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.225038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.257341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1391/2777] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:16 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:16.258230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.259953 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.278116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589deb1-75fe-468e-bf9c-f58c0094f7ea admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.325098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a3b2-337c-40ed-bd72-b490e18f9f2f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.331269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb09b0-76d1-4b01-a248-333561a8e48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.331971 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb09b0-76d1-4b01-a248-333561a8e48b admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.332354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb09b0-76d1-4b01-a248-333561a8e48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.332713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb09b0-76d1-4b01-a248-333561a8e48b admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.333424 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb09b0-76d1-4b01-a248-333561a8e48b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.335348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb09b0-76d1-4b01-a248-333561a8e48b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.339596 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.340363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.340718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.341066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] PATH_INFO: `/v3/users/d421c4d8b73b49d3906bb30fd033e851` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.342295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d421c4d8b73b49d3906bb30fd033e851)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.344363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.349895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:16.357649 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1392/2778] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:16 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:16.362940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb09b0-76d1-4b01-a248-333561a8e48b admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.373823 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb09b0-76d1-4b01-a248-333561a8e48b admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.381534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c428c-9fb5-423b-8733-036972079121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.382192 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c428c-9fb5-423b-8733-036972079121 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.382605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c428c-9fb5-423b-8733-036972079121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.382925 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c428c-9fb5-423b-8733-036972079121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.383636 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c428c-9fb5-423b-8733-036972079121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.385488 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c428c-9fb5-423b-8733-036972079121 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.399838 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1393/2779] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:16.404022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c428c-9fb5-423b-8733-036972079121 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.405461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd421c4d8b73b49d3906bb30fd033e851'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.414844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c428c-9fb5-423b-8733-036972079121 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.415274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd421c4d8b73b49d3906bb30fd033e851'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.416281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd421c4d8b73b49d3906bb30fd033e851'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.420669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.421320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.421598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.421889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.422517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.425012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.432404 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd421c4d8b73b49d3906bb30fd033e851'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.433132 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd421c4d8b73b49d3906bb30fd033e851'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.436569 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1394/2780] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:16.440992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.442705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.447116 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1387/2781] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:16 2022] DELETE /identity/v3/users/d421c4d8b73b49d3906bb30fd033e851 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:16.457606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca758ba2-a9f8-4e3f-a32f-2b5e04e7ea77 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.477791 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:16.517611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.521150 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.544972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce4444-1ac5-4721-9296-6be778c49af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.545644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4444-1ac5-4721-9296-6be778c49af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.546062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4444-1ac5-4721-9296-6be778c49af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.546441 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce4444-1ac5-4721-9296-6be778c49af8 admin admin] PATH_INFO: `/v3/projects/062012c685e0498a9c9b089369d33d88/users/8dab6a88ef324a969f7bad421bbd6972/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.565162 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce4444-1ac5-4721-9296-6be778c49af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=062012c685e0498a9c9b089369d33d88, user_id=8dab6a88ef324a969f7bad421bbd6972, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.567862 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce4444-1ac5-4721-9296-6be778c49af8 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.587312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1395/2782] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:16 2022] PUT /identity/v3/projects/062012c685e0498a9c9b089369d33d88/users/8dab6a88ef324a969f7bad421bbd6972/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:16.597844 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b869-fbc5-416c-9cab-f4f72a3697d2 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.598123 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b869-fbc5-416c-9cab-f4f72a3697d2 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.598397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751b869-fbc5-416c-9cab-f4f72a3697d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.630921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c751b869-fbc5-416c-9cab-f4f72a3697d2 None None] MFA Rules not processed for user `8dab6a88ef324a969f7bad421bbd6972`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:16.632185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c751b869-fbc5-416c-9cab-f4f72a3697d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.652707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1dd84-42c7-49be-8203-e8a820d1f31c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.733062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1396/2783] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:16.741615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.742229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.742509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.742779 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.744771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.747968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751b869-fbc5-416c-9cab-f4f72a3697d2 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.751523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c751b869-fbc5-416c-9cab-f4f72a3697d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.755580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:16.758515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.760311 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.785674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1388/2784] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:16.786718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:16.789466 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:16.847106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751b869-fbc5-416c-9cab-f4f72a3697d2 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.851523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c222c9-f398-459a-b1a4-7028af03e007 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.852136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c222c9-f398-459a-b1a4-7028af03e007 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.852478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c222c9-f398-459a-b1a4-7028af03e007 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.852936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c222c9-f398-459a-b1a4-7028af03e007 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.868551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c222c9-f398-459a-b1a4-7028af03e007 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.870098 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c222c9-f398-459a-b1a4-7028af03e007 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.877079 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf1275-6e3d-4b58-930f-c9395cc3cb6f service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:16.884221 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1397/2785] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:16.888232 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:16.888993 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:16.889351 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:16.889720 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] PATH_INFO: `/v3/projects/1bd852dc9350427ea7c5634932a02965` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:16.891110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd852dc9350427ea7c5634932a02965)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:16.893584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:16.952214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd852dc9350427ea7c5634932a02965', 'request_id': 'req-6554a9fb-32be-4e94-939a-fd23a481ea2b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.960110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd852dc9350427ea7c5634932a02965', 'request_id': 'req-6554a9fb-32be-4e94-939a-fd23a481ea2b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.967336 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd852dc9350427ea7c5634932a02965', 'request_id': 'req-6554a9fb-32be-4e94-939a-fd23a481ea2b'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.985586 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] The token cache is being invalidate because project 1bd852dc9350427ea7c5634932a02965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:16.985995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.986716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:16.988499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1389/2786] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:16 2022] DELETE /identity/v3/projects/1bd852dc9350427ea7c5634932a02965 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:16.992350 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c222c9-f398-459a-b1a4-7028af03e007 service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:17.003747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c222c9-f398-459a-b1a4-7028af03e007 service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:17.046570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:17.087466 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:17.088786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.089247 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.089526 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] PATH_INFO: `/v3/users/253a540ab8f0477eb9ac7414a7644c4d` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.091318 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253a540ab8f0477eb9ac7414a7644c4d)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:17.093592 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:17.157972 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253a540ab8f0477eb9ac7414a7644c4d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.171252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253a540ab8f0477eb9ac7414a7644c4d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.172347 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253a540ab8f0477eb9ac7414a7644c4d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.182142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253a540ab8f0477eb9ac7414a7644c4d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.195903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-19bacc2c-3611-4999-8409-636520b8f733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253a540ab8f0477eb9ac7414a7644c4d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.205368 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1398/2787] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:16 2022] DELETE /identity/v3/users/253a540ab8f0477eb9ac7414a7644c4d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:17.238755 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:17.240335 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:17.255540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6554a9fb-32be-4e94-939a-fd23a481ea2b admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:17.331489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:17.332230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.332512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.332814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] PATH_INFO: `/v3/users/078aa3a96563461696e50ad53343aaef` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.334051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=078aa3a96563461696e50ad53343aaef)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:17.336366 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:17.370979 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:17.411985 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078aa3a96563461696e50ad53343aaef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.422317 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078aa3a96563461696e50ad53343aaef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.423201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078aa3a96563461696e50ad53343aaef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.441924 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078aa3a96563461696e50ad53343aaef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.442709 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078aa3a96563461696e50ad53343aaef'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.456420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1390/2788] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:17 2022] DELETE /identity/v3/users/078aa3a96563461696e50ad53343aaef => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:17.461036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca56bb-13b4-42aa-8652-1af4a820a7f7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.461600 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca56bb-13b4-42aa-8652-1af4a820a7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.461716 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca56bb-13b4-42aa-8652-1af4a820a7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.483148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-90ca56bb-13b4-42aa-8652-1af4a820a7f7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:17.484405 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-90ca56bb-13b4-42aa-8652-1af4a820a7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:17.526272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:17.527767 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:17.553490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1399/2789] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:17.565401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca56bb-13b4-42aa-8652-1af4a820a7f7 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:17.566928 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-90ca56bb-13b4-42aa-8652-1af4a820a7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:17.599816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5ac86-55d2-4bf2-8dc7-0a0cc8341905 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:17.606331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:17.607385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.607673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.607949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] PATH_INFO: `/v3/projects/570143e2306d43fca062f7c0e2d46acf` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.609139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570143e2306d43fca062f7c0e2d46acf)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:17.611140 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:17.635028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca56bb-13b4-42aa-8652-1af4a820a7f7 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:17.638679 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c01a43-1a0e-47c8-b357-5b80350294c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:17.639280 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c01a43-1a0e-47c8-b357-5b80350294c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.639554 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c01a43-1a0e-47c8-b357-5b80350294c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.639818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c01a43-1a0e-47c8-b357-5b80350294c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.650062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c01a43-1a0e-47c8-b357-5b80350294c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:17.651637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c01a43-1a0e-47c8-b357-5b80350294c5 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:17.655896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:17.667224 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570143e2306d43fca062f7c0e2d46acf', 'request_id': 'req-77968946-2848-4c23-9391-8ef83cc7cf06'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.668410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1400/2790] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:04:17.674637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570143e2306d43fca062f7c0e2d46acf', 'request_id': 'req-77968946-2848-4c23-9391-8ef83cc7cf06'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.681429 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570143e2306d43fca062f7c0e2d46acf', 'request_id': 'req-77968946-2848-4c23-9391-8ef83cc7cf06'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.694866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] The token cache is being invalidate because project 570143e2306d43fca062f7c0e2d46acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:17.695159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.695832 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-77968946-2848-4c23-9391-8ef83cc7cf06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:17.697384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1391/2791] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:17 2022] DELETE /identity/v3/projects/570143e2306d43fca062f7c0e2d46acf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:17.703689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0dd75-d256-452b-be2a-8c49244f434c None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.704065 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0dd75-d256-452b-be2a-8c49244f434c None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.705186 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0dd75-d256-452b-be2a-8c49244f434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.726349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-cad0dd75-d256-452b-be2a-8c49244f434c None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:17.727602 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-cad0dd75-d256-452b-be2a-8c49244f434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:17.791171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1401/2792] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:17.801271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a10f1-2928-48c7-a64b-b2e40a0f4c75 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.801651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a10f1-2928-48c7-a64b-b2e40a0f4c75 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.802119 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a10f1-2928-48c7-a64b-b2e40a0f4c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.822973 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-903a10f1-2928-48c7-a64b-b2e40a0f4c75 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:17.824217 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-903a10f1-2928-48c7-a64b-b2e40a0f4c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:17.833549 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1392/2793] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:17.837606 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0dd75-d256-452b-be2a-8c49244f434c None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:17.847557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0dd75-d256-452b-be2a-8c49244f434c None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:17.852447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:17.853062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.853410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.853746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.854493 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:17.854860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:17.856542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:17.870484 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1402/2794] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:04:17.875148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a10f1-2928-48c7-a64b-b2e40a0f4c75 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:17.885052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a10f1-2928-48c7-a64b-b2e40a0f4c75 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:17.890612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:17.891297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:17.891648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:17.892792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:17.894047 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:17.896292 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:17.931436 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1393/2795] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:17 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:17.932797 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:17.943383 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea463882-3e48-4224-bcd8-3bb0a9a4cfd5 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:17.957267 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:17.961445 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:17.976487 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:18.034044 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaed8d-bac2-4568-bcdd-40cd111a6caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:18.035701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaed8d-bac2-4568-bcdd-40cd111a6caa admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.036276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaed8d-bac2-4568-bcdd-40cd111a6caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.036779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfaed8d-bac2-4568-bcdd-40cd111a6caa admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.039550 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaed8d-bac2-4568-bcdd-40cd111a6caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:18.042004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfaed8d-bac2-4568-bcdd-40cd111a6caa admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:18.054456 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b3f58-ae11-4f95-8899-f5985c5ac56d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:18.068233 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:18.069841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.070104 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.070384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] PATH_INFO: `/v3/projects/04b1dc0422554ef89a475d6fa37f53f2` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.073580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b1dc0422554ef89a475d6fa37f53f2)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:18.076593 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:18.085087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1403/2796] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:17 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:18.087343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaed8d-bac2-4568-bcdd-40cd111a6caa admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:18.101025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfaed8d-bac2-4568-bcdd-40cd111a6caa admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:18.108227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6d60f-51a7-4422-9688-13b136cee01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:18.109233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6d60f-51a7-4422-9688-13b136cee01a admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.109748 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6d60f-51a7-4422-9688-13b136cee01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.110308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6d60f-51a7-4422-9688-13b136cee01a admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.112048 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6d60f-51a7-4422-9688-13b136cee01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:18.114177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6d60f-51a7-4422-9688-13b136cee01a admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:18.135093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1404/2797] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:18.136356 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6d60f-51a7-4422-9688-13b136cee01a admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:18.147644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6d60f-51a7-4422-9688-13b136cee01a admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:18.150294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b1dc0422554ef89a475d6fa37f53f2', 'request_id': 'req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.158562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b1dc0422554ef89a475d6fa37f53f2', 'request_id': 'req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.167409 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b1dc0422554ef89a475d6fa37f53f2', 'request_id': 'req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.180996 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] The token cache is being invalidate because project 04b1dc0422554ef89a475d6fa37f53f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:18.181286 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.182554 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.185127 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1394/2798] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:17 2022] DELETE /identity/v3/projects/04b1dc0422554ef89a475d6fa37f53f2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:18.190082 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:18.191560 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:18.225469 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:18.226271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.226557 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.226828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.227491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:18.229144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:18.236607 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:18.246463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1405/2799] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:18.247501 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:18.250250 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:18.263803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0383e-7b2b-4c8a-ab8f-62a11f2f11e8 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:18.264444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb80f0-de74-473d-ad13-683c765c0d96 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:18.269116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab0202-39f6-4947-852a-b9cd5651b1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:18.269388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:18.269793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab0202-39f6-4947-852a-b9cd5651b1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.270000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.270121 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab0202-39f6-4947-852a-b9cd5651b1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.270301 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.270419 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab0202-39f6-4947-852a-b9cd5651b1fb admin admin] PATH_INFO: `/v3/projects/622a759e4e3b48cb93aa1a6b54b38bed/users/19060feb21d24c369c7f1b806e7457b4/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.270531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] PATH_INFO: `/v3/users/3110579258b942ba9468179f48c92c55` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.271790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3110579258b942ba9468179f48c92c55)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:18.273783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:18.287149 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab0202-39f6-4947-852a-b9cd5651b1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=622a759e4e3b48cb93aa1a6b54b38bed, user_id=19060feb21d24c369c7f1b806e7457b4, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:18.289851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab0202-39f6-4947-852a-b9cd5651b1fb admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:18.305727 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1406/2800] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:18 2022] PUT /identity/v3/projects/622a759e4e3b48cb93aa1a6b54b38bed/users/19060feb21d24c369c7f1b806e7457b4/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:18.317239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa449-f21f-4d6b-ba44-61652c750333 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.317876 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa449-f21f-4d6b-ba44-61652c750333 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.318334 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa449-f21f-4d6b-ba44-61652c750333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.324446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:18.332689 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3110579258b942ba9468179f48c92c55'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.342584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3110579258b942ba9468179f48c92c55'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.343219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3110579258b942ba9468179f48c92c55'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.354102 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ad7aa449-f21f-4d6b-ba44-61652c750333 None None] MFA Rules not processed for user `19060feb21d24c369c7f1b806e7457b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:18.355594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ad7aa449-f21f-4d6b-ba44-61652c750333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:18.357630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3110579258b942ba9468179f48c92c55'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.358440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3110579258b942ba9468179f48c92c55'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.371583 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1395/2801] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:18 2022] DELETE /identity/v3/users/3110579258b942ba9468179f48c92c55 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:18.439702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1407/2802] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:18.448525 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:18.450771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:18.520938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8378c77-803c-4ad5-aaf1-709068a831ca admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:18.526808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe0ef4-4607-4af1-a300-045d90f3a8f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7a2237c18934cb2aa3699a5776874ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:18.527603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0ef4-4607-4af1-a300-045d90f3a8f2 service glance] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.528229 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0ef4-4607-4af1-a300-045d90f3a8f2 service glance] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.528543 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0ef4-4607-4af1-a300-045d90f3a8f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.610027 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe0ef4-4607-4af1-a300-045d90f3a8f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:18.612941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe0ef4-4607-4af1-a300-045d90f3a8f2 service glance] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:18.623938 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1396/2803] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:18.709028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa449-f21f-4d6b-ba44-61652c750333 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:18.710562 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ad7aa449-f21f-4d6b-ba44-61652c750333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:18.780060 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa449-f21f-4d6b-ba44-61652c750333 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:18.840522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:18.841155 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.841436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.841712 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] PATH_INFO: `/v3/projects/c2f5f052ee044105b95ae1a2a54d289e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.843292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f5f052ee044105b95ae1a2a54d289e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:18.845761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:18.862434 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:18.894968 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f5f052ee044105b95ae1a2a54d289e', 'request_id': 'req-28ff4739-52e5-4260-8ec6-5f7858cc925b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.901101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f5f052ee044105b95ae1a2a54d289e', 'request_id': 'req-28ff4739-52e5-4260-8ec6-5f7858cc925b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.906618 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f5f052ee044105b95ae1a2a54d289e', 'request_id': 'req-28ff4739-52e5-4260-8ec6-5f7858cc925b'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.920463 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] The token cache is being invalidate because project c2f5f052ee044105b95ae1a2a54d289e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:18.920756 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.921381 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-28ff4739-52e5-4260-8ec6-5f7858cc925b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:18.922866 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1408/2804] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:18 2022] DELETE /identity/v3/projects/c2f5f052ee044105b95ae1a2a54d289e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:18.927922 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc753fc-19be-4674-8eb3-31cc72aa90eb None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:18.928240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc753fc-19be-4674-8eb3-31cc72aa90eb None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:18.928515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc753fc-19be-4674-8eb3-31cc72aa90eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:18.950627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cbc753fc-19be-4674-8eb3-31cc72aa90eb None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:18.952390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cbc753fc-19be-4674-8eb3-31cc72aa90eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:19.019088 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1397/2805] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:19.028020 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba5a9-6191-4fb7-aa91-2031fc0b3385 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.028354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba5a9-6191-4fb7-aa91-2031fc0b3385 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.028644 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba5a9-6191-4fb7-aa91-2031fc0b3385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.050566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-3e0ba5a9-6191-4fb7-aa91-2031fc0b3385 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:19.051788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-3e0ba5a9-6191-4fb7-aa91-2031fc0b3385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:19.075150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1409/2806] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:19.079248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc753fc-19be-4674-8eb3-31cc72aa90eb None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:19.089199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc753fc-19be-4674-8eb3-31cc72aa90eb None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:19.094263 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:19.095531 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.096559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.096559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.096994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:19.097765 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:19.099418 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:19.106871 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:19.115696 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1398/2807] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:04:19.122695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ba5a9-6191-4fb7-aa91-2031fc0b3385 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:19.132343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ba5a9-6191-4fb7-aa91-2031fc0b3385 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:19.136961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa9e88-4956-46a2-9dcf-edd16f537289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:19.137517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa9e88-4956-46a2-9dcf-edd16f537289 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.137792 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa9e88-4956-46a2-9dcf-edd16f537289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.138058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa9e88-4956-46a2-9dcf-edd16f537289 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.138710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa9e88-4956-46a2-9dcf-edd16f537289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:19.140612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa9e88-4956-46a2-9dcf-edd16f537289 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:19.174154 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1410/2808] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:19 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:19.175716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:19.186631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f84d03-c33c-4342-a20f-7e0d26b72c12 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:19.245775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a0e4-0868-4c00-a034-0e943ed4bb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:19.246385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a0e4-0868-4c00-a034-0e943ed4bb02 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.246665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a0e4-0868-4c00-a034-0e943ed4bb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.246940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a0e4-0868-4c00-a034-0e943ed4bb02 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.247563 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1a0e4-0868-4c00-a034-0e943ed4bb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:19.249509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1a0e4-0868-4c00-a034-0e943ed4bb02 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:19.272226 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1399/2809] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:19 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:19.275821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa9e88-4956-46a2-9dcf-edd16f537289 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:19.285998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa9e88-4956-46a2-9dcf-edd16f537289 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:19.290653 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680beb3b-0428-40d3-9dfd-78214b2df69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:19.291254 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680beb3b-0428-40d3-9dfd-78214b2df69d admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.291516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680beb3b-0428-40d3-9dfd-78214b2df69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.291782 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680beb3b-0428-40d3-9dfd-78214b2df69d admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.292451 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680beb3b-0428-40d3-9dfd-78214b2df69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:19.294019 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680beb3b-0428-40d3-9dfd-78214b2df69d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:19.304220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1411/2810] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:19.308021 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a0e4-0868-4c00-a034-0e943ed4bb02 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:19.318054 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a0e4-0868-4c00-a034-0e943ed4bb02 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:19.322649 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1833a9a-3d2c-4009-8685-ed077183d58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:19.323237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1833a9a-3d2c-4009-8685-ed077183d58f admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.323510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1833a9a-3d2c-4009-8685-ed077183d58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.323775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1833a9a-3d2c-4009-8685-ed077183d58f admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.324439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1833a9a-3d2c-4009-8685-ed077183d58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:19.326260 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1833a9a-3d2c-4009-8685-ed077183d58f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:19.336648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1400/2811] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:19.340711 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680beb3b-0428-40d3-9dfd-78214b2df69d admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:19.351179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680beb3b-0428-40d3-9dfd-78214b2df69d admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:19.355859 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:19.356616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.357004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.357281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] PATH_INFO: `/v3/projects/af144f0add0f4e3b9f42734ce682dd59/users/348d838938cb4568b60eef06353536ed/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.372239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af144f0add0f4e3b9f42734ce682dd59, user_id=348d838938cb4568b60eef06353536ed, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:19.374852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:19.389770 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1412/2812] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:19 2022] PUT /identity/v3/projects/af144f0add0f4e3b9f42734ce682dd59/users/348d838938cb4568b60eef06353536ed/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:19.400655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6a9a6-3751-42be-b4ea-df331ce9102a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.400935 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6a9a6-3751-42be-b4ea-df331ce9102a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.401205 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6a9a6-3751-42be-b4ea-df331ce9102a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.431209 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-e6e6a9a6-3751-42be-b4ea-df331ce9102a None None] MFA Rules not processed for user `348d838938cb4568b60eef06353536ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:19.432454 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e6e6a9a6-3751-42be-b4ea-df331ce9102a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:19.501874 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1401/2813] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:19.514291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:19.515984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:19.573462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ad3f-f265-445e-8d1d-63787a55a697 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:19.577291 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18001803-a630-4806-aa2d-d0dfb561840c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:19.577895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18001803-a630-4806-aa2d-d0dfb561840c service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:19.578167 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18001803-a630-4806-aa2d-d0dfb561840c service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:19.578431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18001803-a630-4806-aa2d-d0dfb561840c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:19.587115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18001803-a630-4806-aa2d-d0dfb561840c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:19.588572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18001803-a630-4806-aa2d-d0dfb561840c service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:19.597733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1413/2814] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:26.218505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6a9a6-3751-42be-b4ea-df331ce9102a None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:26.219926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-e6e6a9a6-3751-42be-b4ea-df331ce9102a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:26.287002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6a9a6-3751-42be-b4ea-df331ce9102a None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:26.291708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:26.292400 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:26.292678 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:26.292950 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] PATH_INFO: `/v3/users/8dab6a88ef324a969f7bad421bbd6972` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:26.294090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dab6a88ef324a969f7bad421bbd6972)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:26.296016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:26.302371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:26.345172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dab6a88ef324a969f7bad421bbd6972'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.353156 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dab6a88ef324a969f7bad421bbd6972'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.353930 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dab6a88ef324a969f7bad421bbd6972'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.368278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dab6a88ef324a969f7bad421bbd6972'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.369294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dab6a88ef324a969f7bad421bbd6972'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.383273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1402/2815] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:26 2022] DELETE /identity/v3/users/8dab6a88ef324a969f7bad421bbd6972 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:26.388043 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f858d7-7f3d-475f-9633-0863e13a1c79 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:26.388584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f858d7-7f3d-475f-9633-0863e13a1c79 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:26.388695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f858d7-7f3d-475f-9633-0863e13a1c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:26.408819 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c1f858d7-7f3d-475f-9633-0863e13a1c79 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:26.410429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c1f858d7-7f3d-475f-9633-0863e13a1c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:26.449903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:26.474878 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1414/2816] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:26.484478 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:26.485915 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:26.545485 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f4cb8-0958-4935-ad5e-a03f789b7550 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:26.549186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db7ab5-afcb-4847-82fd-03be066e10f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:26.549801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db7ab5-afcb-4847-82fd-03be066e10f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:26.550078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db7ab5-afcb-4847-82fd-03be066e10f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:26.550348 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db7ab5-afcb-4847-82fd-03be066e10f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:26.566327 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db7ab5-afcb-4847-82fd-03be066e10f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:26.568141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db7ab5-afcb-4847-82fd-03be066e10f8 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:26.579871 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1403/2817] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:26.859519 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f858d7-7f3d-475f-9633-0863e13a1c79 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:26.862176 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c1f858d7-7f3d-475f-9633-0863e13a1c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:26.877952 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f858d7-7f3d-475f-9633-0863e13a1c79 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:26.882892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:26.883536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:26.883815 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:26.884086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] PATH_INFO: `/v3/projects/062012c685e0498a9c9b089369d33d88` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:26.885238 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] RBAC: Authorizing `identity:delete_project(project_id=062012c685e0498a9c9b089369d33d88)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:26.887179 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:26.942860 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '062012c685e0498a9c9b089369d33d88', 'request_id': 'req-b0673877-58bf-48a7-9145-e84e101e653d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.949112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '062012c685e0498a9c9b089369d33d88', 'request_id': 'req-b0673877-58bf-48a7-9145-e84e101e653d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.954889 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '062012c685e0498a9c9b089369d33d88', 'request_id': 'req-b0673877-58bf-48a7-9145-e84e101e653d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.967286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] The token cache is being invalidate because project 062012c685e0498a9c9b089369d33d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:26.967572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.968216 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-b0673877-58bf-48a7-9145-e84e101e653d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:26.969685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1415/2818] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:26 2022] DELETE /identity/v3/projects/062012c685e0498a9c9b089369d33d88 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:26.975264 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe7295-6bd0-4c16-a0e8-3181337ff6a9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:26.975550 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe7295-6bd0-4c16-a0e8-3181337ff6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:26.975825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe7295-6bd0-4c16-a0e8-3181337ff6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:26.997514 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-16fe7295-6bd0-4c16-a0e8-3181337ff6a9 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:26.998954 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-16fe7295-6bd0-4c16-a0e8-3181337ff6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:27.062680 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1404/2819] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:27.128299 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0698171-71c6-4416-8ac0-24786b93ac6f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.128547 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0698171-71c6-4416-8ac0-24786b93ac6f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.128757 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0698171-71c6-4416-8ac0-24786b93ac6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.148612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-e0698171-71c6-4416-8ac0-24786b93ac6f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:27.149828 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-e0698171-71c6-4416-8ac0-24786b93ac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:27.157755 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1416/2820] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:27.162677 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe7295-6bd0-4c16-a0e8-3181337ff6a9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:27.172333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe7295-6bd0-4c16-a0e8-3181337ff6a9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:27.177465 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:27.178058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.178333 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.178598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.179665 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:27.179967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:27.181590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:27.193306 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1405/2821] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:04:27.198063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0698171-71c6-4416-8ac0-24786b93ac6f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:27.208402 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0698171-71c6-4416-8ac0-24786b93ac6f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:27.213054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a51c-3bc6-4221-a901-79418f09d94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:27.213652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a51c-3bc6-4221-a901-79418f09d94c admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.213927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a51c-3bc6-4221-a901-79418f09d94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.214193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6a51c-3bc6-4221-a901-79418f09d94c admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.214813 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6a51c-3bc6-4221-a901-79418f09d94c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:27.216745 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6a51c-3bc6-4221-a901-79418f09d94c admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:27.243929 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1417/2822] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:27 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:27.248124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:27.259473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b91dd-8765-4492-a460-9cec39e0ca64 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:27.275034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:27.323126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c4c37-7216-46f2-92dd-9defeacc3202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:27.323730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c4c37-7216-46f2-92dd-9defeacc3202 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.324008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c4c37-7216-46f2-92dd-9defeacc3202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.324310 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c4c37-7216-46f2-92dd-9defeacc3202 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.324926 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c4c37-7216-46f2-92dd-9defeacc3202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:27.327092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c4c37-7216-46f2-92dd-9defeacc3202 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:27.349735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1406/2823] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:27 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:27.353697 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a51c-3bc6-4221-a901-79418f09d94c admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:27.364229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6a51c-3bc6-4221-a901-79418f09d94c admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:27.368778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce78cf2-b7e5-4ecb-813c-205fad55eb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:27.369362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce78cf2-b7e5-4ecb-813c-205fad55eb53 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.369637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce78cf2-b7e5-4ecb-813c-205fad55eb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.369902 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce78cf2-b7e5-4ecb-813c-205fad55eb53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.370538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce78cf2-b7e5-4ecb-813c-205fad55eb53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:27.372133 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce78cf2-b7e5-4ecb-813c-205fad55eb53 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:27.382301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1418/2824] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:27.386298 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c4c37-7216-46f2-92dd-9defeacc3202 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:27.398259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c4c37-7216-46f2-92dd-9defeacc3202 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:27.402998 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dcf89-5c8b-4c8f-be5a-bd40000c19f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:27.403601 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dcf89-5c8b-4c8f-be5a-bd40000c19f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.403875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dcf89-5c8b-4c8f-be5a-bd40000c19f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.404172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dcf89-5c8b-4c8f-be5a-bd40000c19f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.404813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dcf89-5c8b-4c8f-be5a-bd40000c19f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:27.406517 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dcf89-5c8b-4c8f-be5a-bd40000c19f8 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:27.418078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1407/2825] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:27.422268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce78cf2-b7e5-4ecb-813c-205fad55eb53 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:27.433066 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce78cf2-b7e5-4ecb-813c-205fad55eb53 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:27.438490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:27.439944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.440281 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.440523 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] PATH_INFO: `/v3/projects/e75db9cf63624704afe00066b973360b/users/99baf25bdb4340b8a7947f08e426bd46/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.457308 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e75db9cf63624704afe00066b973360b, user_id=99baf25bdb4340b8a7947f08e426bd46, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:27.460012 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:27.472533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1419/2826] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:27 2022] PUT /identity/v3/projects/e75db9cf63624704afe00066b973360b/users/99baf25bdb4340b8a7947f08e426bd46/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:27.477534 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffac28-047e-41da-8dd9-b51e71aee75a None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.477846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffac28-047e-41da-8dd9-b51e71aee75a None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.478120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffac28-047e-41da-8dd9-b51e71aee75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.497762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-efffac28-047e-41da-8dd9-b51e71aee75a None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:27.499034 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-efffac28-047e-41da-8dd9-b51e71aee75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:27.558734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1408/2827] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:27.569030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:27.570754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:27.646470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7d643-68a3-44cd-bbcd-6f77ef19a884 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:27.650205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c32c-f0a5-4d5f-a5a0-b74e46c542fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:27.650818 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c32c-f0a5-4d5f-a5a0-b74e46c542fe service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:27.651099 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c32c-f0a5-4d5f-a5a0-b74e46c542fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:27.651369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03c32c-f0a5-4d5f-a5a0-b74e46c542fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:27.660728 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03c32c-f0a5-4d5f-a5a0-b74e46c542fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:27.662157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03c32c-f0a5-4d5f-a5a0-b74e46c542fe service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:27.671500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1420/2828] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:31.831032 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f565-edf9-4544-bfe7-57be547a44b1 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:31.831610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f565-edf9-4544-bfe7-57be547a44b1 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:31.831610 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118f565-edf9-4544-bfe7-57be547a44b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:31.868489 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-1118f565-edf9-4544-bfe7-57be547a44b1 None None] MFA Rules not processed for user `99baf25bdb4340b8a7947f08e426bd46`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:31.869746 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1118f565-edf9-4544-bfe7-57be547a44b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:31.947625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1409/2829] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:31.953586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c32c-f0a5-4d5f-a5a0-b74e46c542fe service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:31.962457 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:31.966485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03c32c-f0a5-4d5f-a5a0-b74e46c542fe service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:31.973500 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf750d2-a3f0-4dfd-9a82-ebe9a3d39933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:31.974289 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf750d2-a3f0-4dfd-9a82-ebe9a3d39933 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:31.974905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf750d2-a3f0-4dfd-9a82-ebe9a3d39933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:31.975322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf750d2-a3f0-4dfd-9a82-ebe9a3d39933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:31.976266 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf750d2-a3f0-4dfd-9a82-ebe9a3d39933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:31.978193 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf750d2-a3f0-4dfd-9a82-ebe9a3d39933 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:32.009975 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1421/2830] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:31 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:32.014075 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118f565-edf9-4544-bfe7-57be547a44b1 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:32.026463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118f565-edf9-4544-bfe7-57be547a44b1 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:32.051895 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:32.094937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e9af8-9bb1-4ebb-83a2-8dc9eb25dea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:32.095997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e9af8-9bb1-4ebb-83a2-8dc9eb25dea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:32.096569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e9af8-9bb1-4ebb-83a2-8dc9eb25dea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:32.096941 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e9af8-9bb1-4ebb-83a2-8dc9eb25dea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:32.097757 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e9af8-9bb1-4ebb-83a2-8dc9eb25dea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:32.100008 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e9af8-9bb1-4ebb-83a2-8dc9eb25dea9 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:32.123110 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1410/2831] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:32 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:32.127877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf750d2-a3f0-4dfd-9a82-ebe9a3d39933 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:32.139028 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf750d2-a3f0-4dfd-9a82-ebe9a3d39933 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:32.144897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369c5d8-3ae2-4d88-9c39-2577808aba54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:32.145511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369c5d8-3ae2-4d88-9c39-2577808aba54 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:32.145794 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369c5d8-3ae2-4d88-9c39-2577808aba54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:32.146054 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369c5d8-3ae2-4d88-9c39-2577808aba54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:32.146750 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369c5d8-3ae2-4d88-9c39-2577808aba54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:32.148422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369c5d8-3ae2-4d88-9c39-2577808aba54 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:32.160847 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1422/2832] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:32.164461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e9af8-9bb1-4ebb-83a2-8dc9eb25dea9 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:32.179636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e9af8-9bb1-4ebb-83a2-8dc9eb25dea9 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:32.184628 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe799a2a-55a3-4a11-8a9e-cbc9cec4c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:32.185323 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe799a2a-55a3-4a11-8a9e-cbc9cec4c1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:32.185605 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe799a2a-55a3-4a11-8a9e-cbc9cec4c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:32.185887 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe799a2a-55a3-4a11-8a9e-cbc9cec4c1e5 admin admin] PATH_INFO: `/v3/projects/48cc9eed717b48baa3853b5c4cbf33f3/users/f9a1308590114889884ba944d0b5a105/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:32.206128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe799a2a-55a3-4a11-8a9e-cbc9cec4c1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cc9eed717b48baa3853b5c4cbf33f3, user_id=f9a1308590114889884ba944d0b5a105, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:32.210523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe799a2a-55a3-4a11-8a9e-cbc9cec4c1e5 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:32.231889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1411/2833] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:32 2022] PUT /identity/v3/projects/48cc9eed717b48baa3853b5c4cbf33f3/users/f9a1308590114889884ba944d0b5a105/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:32.235678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369c5d8-3ae2-4d88-9c39-2577808aba54 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:32.250209 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369c5d8-3ae2-4d88-9c39-2577808aba54 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:32.310899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:32.311537 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:32.311910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:32.312274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:32.312961 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:32.314709 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:32.324576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1423/2834] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:32.329136 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe799a2a-55a3-4a11-8a9e-cbc9cec4c1e5 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:32.341500 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe799a2a-55a3-4a11-8a9e-cbc9cec4c1e5 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:32.352128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53433807-9044-4738-be25-68a3fe2f9b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:32.352940 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433807-9044-4738-be25-68a3fe2f9b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:32.353297 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433807-9044-4738-be25-68a3fe2f9b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:32.353647 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433807-9044-4738-be25-68a3fe2f9b73 admin admin] PATH_INFO: `/v3/projects/48cc9eed717b48baa3853b5c4cbf33f3/users/f9a1308590114889884ba944d0b5a105/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:32.355833 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53433807-9044-4738-be25-68a3fe2f9b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cc9eed717b48baa3853b5c4cbf33f3, user_id=f9a1308590114889884ba944d0b5a105, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:32.358558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53433807-9044-4738-be25-68a3fe2f9b73 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:32.369700 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1412/2835] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:32 2022] PUT /identity/v3/projects/48cc9eed717b48baa3853b5c4cbf33f3/users/f9a1308590114889884ba944d0b5a105/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:37.523778 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:37.531784 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:37.613991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278a2e1-fe6c-4360-843e-100c25710a02 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:37.614481 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278a2e1-fe6c-4360-843e-100c25710a02 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:37.614589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278a2e1-fe6c-4360-843e-100c25710a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:37.638228 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c446e-e591-49e1-8099-327b0a1a09ef admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:37.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-d278a2e1-fe6c-4360-843e-100c25710a02 None None] MFA Rules not processed for user `f9a1308590114889884ba944d0b5a105`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:37.660170 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d278a2e1-fe6c-4360-843e-100c25710a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:37.707565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:37.708363 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:37.708654 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:37.708927 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:37.710985 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:37.726701 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:37.728821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:37.741473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1424/2836] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:37.742608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:37.771957 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1413/2837] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:37.772517 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447b5e9-5177-4d5f-a7ed-ca78a4112dc7 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:37.777694 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec1d2a-27c8-40db-bbaa-40820780ac31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:37.778561 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec1d2a-27c8-40db-bbaa-40820780ac31 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:37.778848 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec1d2a-27c8-40db-bbaa-40820780ac31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:37.779124 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec1d2a-27c8-40db-bbaa-40820780ac31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:37.781236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-fcec1d2a-27c8-40db-bbaa-40820780ac31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:37.793126 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278a2e1-fe6c-4360-843e-100c25710a02 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:37.795141 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d278a2e1-fe6c-4360-843e-100c25710a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:37.801768 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:37.900255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:37.906295 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec1d2a-27c8-40db-bbaa-40820780ac31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:37.908038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec1d2a-27c8-40db-bbaa-40820780ac31 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:37.927243 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278a2e1-fe6c-4360-843e-100c25710a02 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:37.933100 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1425/2838] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:37.936853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:37.936853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:37.936853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:37.936853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:38.057102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:38.058819 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:38.075503 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1414/2839] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:39.577202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f8e8d-43f7-4d05-9bb5-21231e9f0fe1 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:39.578083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f8e8d-43f7-4d05-9bb5-21231e9f0fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:39.578556 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f8e8d-43f7-4d05-9bb5-21231e9f0fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:39.601731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-f65f8e8d-43f7-4d05-9bb5-21231e9f0fe1 None None] MFA Rules not processed for user `99baf25bdb4340b8a7947f08e426bd46`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:39.603017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f65f8e8d-43f7-4d05-9bb5-21231e9f0fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:39.615588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1426/2840] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:39.954894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:39.956651 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:39.974658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3526b-f1b5-409b-b779-e677d92a4cbb service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:39.983735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ca631-5c65-4f6a-a3bc-7854959f1474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:39.984709 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ca631-5c65-4f6a-a3bc-7854959f1474 service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:39.985237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ca631-5c65-4f6a-a3bc-7854959f1474 service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:39.985747 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ca631-5c65-4f6a-a3bc-7854959f1474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:39.989603 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-291ca631-5c65-4f6a-a3bc-7854959f1474 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:40.007886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ca631-5c65-4f6a-a3bc-7854959f1474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:40.009675 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ca631-5c65-4f6a-a3bc-7854959f1474 service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:40.036271 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1415/2841] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:40.614671 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f8e8d-43f7-4d05-9bb5-21231e9f0fe1 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:40.616222 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-f65f8e8d-43f7-4d05-9bb5-21231e9f0fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:40.818713 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f8e8d-43f7-4d05-9bb5-21231e9f0fe1 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:40.832896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:40.834246 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:40.834538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:40.834814 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] PATH_INFO: `/v3/users/348d838938cb4568b60eef06353536ed` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:40.844635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348d838938cb4568b60eef06353536ed)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:40.849354 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:40.979386 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348d838938cb4568b60eef06353536ed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:41.002879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348d838938cb4568b60eef06353536ed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:41.004062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348d838938cb4568b60eef06353536ed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:41.014161 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348d838938cb4568b60eef06353536ed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:41.027397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348d838938cb4568b60eef06353536ed'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:41.040490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1427/2842] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:40 2022] DELETE /identity/v3/users/348d838938cb4568b60eef06353536ed => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:41.042277 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733df4-c85c-43cd-a0ad-aaa7c82f9f60 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:41.042818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733df4-c85c-43cd-a0ad-aaa7c82f9f60 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:41.042818 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30733df4-c85c-43cd-a0ad-aaa7c82f9f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:41.089314 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-30733df4-c85c-43cd-a0ad-aaa7c82f9f60 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:41.090584 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30733df4-c85c-43cd-a0ad-aaa7c82f9f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:41.229878 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1416/2843] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:41.247144 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:41.249783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:41.300319 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:41.351006 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cd9ac-580d-4ffb-b499-328b46825b16 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:41.357101 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a1e25-241d-4f0f-ab82-248c0693e13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:41.357957 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1e25-241d-4f0f-ab82-248c0693e13c service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:41.358426 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1e25-241d-4f0f-ab82-248c0693e13c service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:41.359436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1e25-241d-4f0f-ab82-248c0693e13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:41.394430 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a1e25-241d-4f0f-ab82-248c0693e13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:41.396767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a1e25-241d-4f0f-ab82-248c0693e13c service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:41.423854 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1428/2844] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:41.905907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733df4-c85c-43cd-a0ad-aaa7c82f9f60 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:41.907685 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-30733df4-c85c-43cd-a0ad-aaa7c82f9f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:41.923997 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30733df4-c85c-43cd-a0ad-aaa7c82f9f60 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:41.929492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:41.930211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:41.930893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:41.931398 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] PATH_INFO: `/v3/projects/af144f0add0f4e3b9f42734ce682dd59` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:41.932695 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af144f0add0f4e3b9f42734ce682dd59)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:41.934768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:42.005045 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af144f0add0f4e3b9f42734ce682dd59', 'request_id': 'req-8c424985-52d0-4c92-b995-7e6758b63347'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:42.014528 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af144f0add0f4e3b9f42734ce682dd59', 'request_id': 'req-8c424985-52d0-4c92-b995-7e6758b63347'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:42.022627 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af144f0add0f4e3b9f42734ce682dd59', 'request_id': 'req-8c424985-52d0-4c92-b995-7e6758b63347'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:42.037181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] The token cache is being invalidate because project af144f0add0f4e3b9f42734ce682dd59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:42.037475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:42.038455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-8c424985-52d0-4c92-b995-7e6758b63347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:42.040385 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1417/2845] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:41 2022] DELETE /identity/v3/projects/af144f0add0f4e3b9f42734ce682dd59 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:42.045699 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70582e66-19a5-41af-8904-205a4367992b None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.046566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70582e66-19a5-41af-8904-205a4367992b None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.047025 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70582e66-19a5-41af-8904-205a4367992b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.070960 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-70582e66-19a5-41af-8904-205a4367992b None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:42.072233 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-70582e66-19a5-41af-8904-205a4367992b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:42.159570 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1429/2846] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:42.161107 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59844cca-321a-485e-b5b2-b7d17ee76e95 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.161614 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59844cca-321a-485e-b5b2-b7d17ee76e95 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.161886 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59844cca-321a-485e-b5b2-b7d17ee76e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.184590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-59844cca-321a-485e-b5b2-b7d17ee76e95 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:42.185843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-59844cca-321a-485e-b5b2-b7d17ee76e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:42.209822 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1418/2847] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:42.211891 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70582e66-19a5-41af-8904-205a4367992b None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:42.234143 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70582e66-19a5-41af-8904-205a4367992b None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:42.247062 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:42.247798 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.248330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.248678 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] PATH_INFO: `/v3/domains` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.249535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:42.249939 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:42.252304 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:42.271480 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1430/2848] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:04:42.272704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59844cca-321a-485e-b5b2-b7d17ee76e95 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:42.280637 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:42.292989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59844cca-321a-485e-b5b2-b7d17ee76e95 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:42.304041 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353300f-d996-44e8-ba69-6932e58cdf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:42.304933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353300f-d996-44e8-ba69-6932e58cdf68 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.305403 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353300f-d996-44e8-ba69-6932e58cdf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.305803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353300f-d996-44e8-ba69-6932e58cdf68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.306776 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353300f-d996-44e8-ba69-6932e58cdf68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:42.309898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353300f-d996-44e8-ba69-6932e58cdf68 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:42.346083 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1419/2849] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:42 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:42.350696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:42.361087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29248e1-34a6-4935-b9bd-05fc6d658607 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:42.449732 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a1762-270d-4907-8ad4-725263bc41bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:42.450825 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a1762-270d-4907-8ad4-725263bc41bd admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.451243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a1762-270d-4907-8ad4-725263bc41bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.452002 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a1762-270d-4907-8ad4-725263bc41bd admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.452747 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a1762-270d-4907-8ad4-725263bc41bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:42.454909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a1762-270d-4907-8ad4-725263bc41bd admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:42.505278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1431/2850] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:42 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:42.510913 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353300f-d996-44e8-ba69-6932e58cdf68 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:42.524450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353300f-d996-44e8-ba69-6932e58cdf68 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:42.535855 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f470a63-bf43-42ec-8aa5-2c86dd11a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:42.536865 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470a63-bf43-42ec-8aa5-2c86dd11a8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.537371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470a63-bf43-42ec-8aa5-2c86dd11a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.537890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f470a63-bf43-42ec-8aa5-2c86dd11a8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.539007 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f470a63-bf43-42ec-8aa5-2c86dd11a8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:42.542827 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f470a63-bf43-42ec-8aa5-2c86dd11a8c2 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:42.557051 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1420/2851] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:42.560726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a1762-270d-4907-8ad4-725263bc41bd admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:42.572376 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a1762-270d-4907-8ad4-725263bc41bd admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:42.581518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d37093-1387-4521-bac7-4b588710b2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:42.582150 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d37093-1387-4521-bac7-4b588710b2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.582429 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d37093-1387-4521-bac7-4b588710b2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.582696 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d37093-1387-4521-bac7-4b588710b2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.583395 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d37093-1387-4521-bac7-4b588710b2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:42.585081 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d37093-1387-4521-bac7-4b588710b2f2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:42.599122 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1432/2852] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:42.603324 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f470a63-bf43-42ec-8aa5-2c86dd11a8c2 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:42.614120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f470a63-bf43-42ec-8aa5-2c86dd11a8c2 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:42.619792 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be05ed-891d-4b70-882d-cff90b7ee1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:42.620559 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be05ed-891d-4b70-882d-cff90b7ee1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.620841 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be05ed-891d-4b70-882d-cff90b7ee1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.621114 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be05ed-891d-4b70-882d-cff90b7ee1bd admin admin] PATH_INFO: `/v3/projects/1065ceba253f44a888732c412e9f1633/users/b09153473a954bd5becb7f65a39b8b57/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.643808 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be05ed-891d-4b70-882d-cff90b7ee1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1065ceba253f44a888732c412e9f1633, user_id=b09153473a954bd5becb7f65a39b8b57, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:42.649951 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be05ed-891d-4b70-882d-cff90b7ee1bd admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:42.674962 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1421/2853] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:42 2022] PUT /identity/v3/projects/1065ceba253f44a888732c412e9f1633/users/b09153473a954bd5becb7f65a39b8b57/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:42.684542 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9f72-e3ed-4959-b85e-c901127fa3cd None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.684832 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9f72-e3ed-4959-b85e-c901127fa3cd None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.685106 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a9f72-e3ed-4959-b85e-c901127fa3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.716586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-4e9a9f72-e3ed-4959-b85e-c901127fa3cd None None] MFA Rules not processed for user `b09153473a954bd5becb7f65a39b8b57`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:42.717840 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4e9a9f72-e3ed-4959-b85e-c901127fa3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:42.746622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:42.799643 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1433/2854] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:42.804395 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be05ed-891d-4b70-882d-cff90b7ee1bd admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:42.870462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be05ed-891d-4b70-882d-cff90b7ee1bd admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:42.875167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f8f67-bc3b-49be-ba92-c12b84da9a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:42.875772 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f8f67-bc3b-49be-ba92-c12b84da9a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.876052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f8f67-bc3b-49be-ba92-c12b84da9a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.876354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f8f67-bc3b-49be-ba92-c12b84da9a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.877521 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f8f67-bc3b-49be-ba92-c12b84da9a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:42.879416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f8f67-bc3b-49be-ba92-c12b84da9a39 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:42.906934 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1422/2855] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:42 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 11 17:04:42.914307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a9f72-e3ed-4959-b85e-c901127fa3cd None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:42.926682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a9f72-e3ed-4959-b85e-c901127fa3cd None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:42.989178 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee964c42-d3ca-4c8e-9d9f-62bedbfc0065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:42.989790 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee964c42-d3ca-4c8e-9d9f-62bedbfc0065 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:42.990069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee964c42-d3ca-4c8e-9d9f-62bedbfc0065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:42.990340 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee964c42-d3ca-4c8e-9d9f-62bedbfc0065 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:42.990964 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee964c42-d3ca-4c8e-9d9f-62bedbfc0065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:42.992905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee964c42-d3ca-4c8e-9d9f-62bedbfc0065 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.017923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1434/2856] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:42 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:43.021204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f8f67-bc3b-49be-ba92-c12b84da9a39 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.031670 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f8f67-bc3b-49be-ba92-c12b84da9a39 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.036374 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7027157-0ede-46a1-84c3-b97f3ec604c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.037014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7027157-0ede-46a1-84c3-b97f3ec604c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.037295 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7027157-0ede-46a1-84c3-b97f3ec604c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.037569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7027157-0ede-46a1-84c3-b97f3ec604c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.038210 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7027157-0ede-46a1-84c3-b97f3ec604c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.039837 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7027157-0ede-46a1-84c3-b97f3ec604c4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.054565 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1423/2857] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:43.058590 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee964c42-d3ca-4c8e-9d9f-62bedbfc0065 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.069397 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee964c42-d3ca-4c8e-9d9f-62bedbfc0065 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.073987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b897e-108b-4bb5-ae4c-b6a3c7b592fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.074576 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b897e-108b-4bb5-ae4c-b6a3c7b592fd admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.074855 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b897e-108b-4bb5-ae4c-b6a3c7b592fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.075126 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b897e-108b-4bb5-ae4c-b6a3c7b592fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.075764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b897e-108b-4bb5-ae4c-b6a3c7b592fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.077651 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b897e-108b-4bb5-ae4c-b6a3c7b592fd admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.087436 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1435/2858] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:43.091304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7027157-0ede-46a1-84c3-b97f3ec604c4 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.101804 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7027157-0ede-46a1-84c3-b97f3ec604c4 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.106929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016111-ddb9-4abb-8327-17f3d57068aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.107643 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016111-ddb9-4abb-8327-17f3d57068aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.107949 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016111-ddb9-4abb-8327-17f3d57068aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.108274 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe016111-ddb9-4abb-8327-17f3d57068aa admin admin] PATH_INFO: `/v3/projects/e6a4c0048db24c80a9fda38fe12dd88e/users/ad2be2f880a04b779f1299c9f94059e4/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.125066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016111-ddb9-4abb-8327-17f3d57068aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a4c0048db24c80a9fda38fe12dd88e, user_id=ad2be2f880a04b779f1299c9f94059e4, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.128272 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe016111-ddb9-4abb-8327-17f3d57068aa admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.140722 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1424/2859] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:43 2022] PUT /identity/v3/projects/e6a4c0048db24c80a9fda38fe12dd88e/users/ad2be2f880a04b779f1299c9f94059e4/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:43.152473 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caab9d-b5f8-4730-a622-2a67ebac55c3 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.152753 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caab9d-b5f8-4730-a622-2a67ebac55c3 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.153022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caab9d-b5f8-4730-a622-2a67ebac55c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.182022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-c6caab9d-b5f8-4730-a622-2a67ebac55c3 None None] MFA Rules not processed for user `ad2be2f880a04b779f1299c9f94059e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:43.183300 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c6caab9d-b5f8-4730-a622-2a67ebac55c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:43.255248 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1436/2860] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:43.259828 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016111-ddb9-4abb-8327-17f3d57068aa admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.323736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:43.331439 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe016111-ddb9-4abb-8327-17f3d57068aa admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.336889 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdd081-8ea5-450d-9ff0-3035bd860958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.337813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdd081-8ea5-450d-9ff0-3035bd860958 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.338304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdd081-8ea5-450d-9ff0-3035bd860958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.339076 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdd081-8ea5-450d-9ff0-3035bd860958 admin admin] PATH_INFO: `/v3/projects` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.339814 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdd081-8ea5-450d-9ff0-3035bd860958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.341849 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdd081-8ea5-450d-9ff0-3035bd860958 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.372623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1425/2861] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:43 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:43.374540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6caab9d-b5f8-4730-a622-2a67ebac55c3 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.386088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6caab9d-b5f8-4730-a622-2a67ebac55c3 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.416997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:43.450312 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4125b-6157-40b6-9265-e95171807365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.451893 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4125b-6157-40b6-9265-e95171807365 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.452264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4125b-6157-40b6-9265-e95171807365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.452619 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4125b-6157-40b6-9265-e95171807365 admin admin] PATH_INFO: `/v3/users` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.453330 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4125b-6157-40b6-9265-e95171807365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.455861 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4125b-6157-40b6-9265-e95171807365 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.479331 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1437/2862] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:43 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:43.483291 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdd081-8ea5-450d-9ff0-3035bd860958 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.497948 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdd081-8ea5-450d-9ff0-3035bd860958 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.504598 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec917b-b261-4d51-ae11-762c0a936691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.505354 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec917b-b261-4d51-ae11-762c0a936691 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.505761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec917b-b261-4d51-ae11-762c0a936691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.506157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec917b-b261-4d51-ae11-762c0a936691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.507002 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec917b-b261-4d51-ae11-762c0a936691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.509078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec917b-b261-4d51-ae11-762c0a936691 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.521473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1426/2863] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:43.525776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4125b-6157-40b6-9265-e95171807365 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.545063 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4125b-6157-40b6-9265-e95171807365 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.551140 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50bdcf-29bd-4a61-8302-d5d28bf7c180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.552082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50bdcf-29bd-4a61-8302-d5d28bf7c180 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.552599 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50bdcf-29bd-4a61-8302-d5d28bf7c180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.552942 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50bdcf-29bd-4a61-8302-d5d28bf7c180 admin admin] PATH_INFO: `/v3/projects/160e5ad3d0694a95a2f89a047a65599c/users/afaa1e41df9143858242d6d0500dd639/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.576171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50bdcf-29bd-4a61-8302-d5d28bf7c180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=160e5ad3d0694a95a2f89a047a65599c, user_id=afaa1e41df9143858242d6d0500dd639, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.576171 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50bdcf-29bd-4a61-8302-d5d28bf7c180 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.590173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1438/2864] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:43 2022] PUT /identity/v3/projects/160e5ad3d0694a95a2f89a047a65599c/users/afaa1e41df9143858242d6d0500dd639/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:43.594017 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec917b-b261-4d51-ae11-762c0a936691 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.605024 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec917b-b261-4d51-ae11-762c0a936691 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.670515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3c780-1c5d-4b19-bd63-59ab34534dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.671300 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c780-1c5d-4b19-bd63-59ab34534dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.671581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c780-1c5d-4b19-bd63-59ab34534dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.671853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c780-1c5d-4b19-bd63-59ab34534dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.672529 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3c780-1c5d-4b19-bd63-59ab34534dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.674656 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3c780-1c5d-4b19-bd63-59ab34534dc4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.684671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1427/2865] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:43.688926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50bdcf-29bd-4a61-8302-d5d28bf7c180 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.703508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50bdcf-29bd-4a61-8302-d5d28bf7c180 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.709933 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.710460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.710738 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.711038 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] PATH_INFO: `/v3/projects/160e5ad3d0694a95a2f89a047a65599c/users/afaa1e41df9143858242d6d0500dd639/roles/29fd031edd2b4f65aed8942853150f92` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.713205 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=160e5ad3d0694a95a2f89a047a65599c, user_id=afaa1e41df9143858242d6d0500dd639, role_id=29fd031edd2b4f65aed8942853150f92)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.715857 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.727723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1439/2866] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:43 2022] PUT /identity/v3/projects/160e5ad3d0694a95a2f89a047a65599c/users/afaa1e41df9143858242d6d0500dd639/roles/29fd031edd2b4f65aed8942853150f92 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:43.738920 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27ec2e-3b33-4b00-826e-3423f9d5ba37 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.739206 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27ec2e-3b33-4b00-826e-3423f9d5ba37 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.739479 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27ec2e-3b33-4b00-826e-3423f9d5ba37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.768444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5c27ec2e-3b33-4b00-826e-3423f9d5ba37 None None] MFA Rules not processed for user `afaa1e41df9143858242d6d0500dd639`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:43.769669 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5c27ec2e-3b33-4b00-826e-3423f9d5ba37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:43.841505 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1428/2867] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:43.852918 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:43.854648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:43.919980 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53e8ff-496f-4a31-917b-c6ca911c7ff8 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:43.923704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a499e1a-8986-4677-9655-f988ff2b006d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:43.924339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a499e1a-8986-4677-9655-f988ff2b006d service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:43.924621 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a499e1a-8986-4677-9655-f988ff2b006d service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:43.924911 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a499e1a-8986-4677-9655-f988ff2b006d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:43.986992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a499e1a-8986-4677-9655-f988ff2b006d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:43.988511 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a499e1a-8986-4677-9655-f988ff2b006d service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:43.998089 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1440/2868] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:46.589312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27ec2e-3b33-4b00-826e-3423f9d5ba37 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:46.590796 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5c27ec2e-3b33-4b00-826e-3423f9d5ba37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:46.602760 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27ec2e-3b33-4b00-826e-3423f9d5ba37 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:46.606284 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7ae7b-7325-4b4b-809a-6d3f002285b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:46.606876 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7ae7b-7325-4b4b-809a-6d3f002285b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:46.607148 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7ae7b-7325-4b4b-809a-6d3f002285b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:46.607416 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7ae7b-7325-4b4b-809a-6d3f002285b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:46.663003 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:46.674564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7ae7b-7325-4b4b-809a-6d3f002285b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:46.675981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7ae7b-7325-4b4b-809a-6d3f002285b4 service cinder] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:46.684813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1429/2869] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:47.270718 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a499e1a-8986-4677-9655-f988ff2b006d service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:47.279252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a499e1a-8986-4677-9655-f988ff2b006d service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:47.283437 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feb1a4-3925-47f9-bfc3-8e35cb87688d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:47.284080 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feb1a4-3925-47f9-bfc3-8e35cb87688d service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:47.284450 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feb1a4-3925-47f9-bfc3-8e35cb87688d service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:47.284783 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feb1a4-3925-47f9-bfc3-8e35cb87688d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:47.293560 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feb1a4-3925-47f9-bfc3-8e35cb87688d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:47.294955 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feb1a4-3925-47f9-bfc3-8e35cb87688d service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:47.304612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1441/2870] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:54.374499 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7ae7b-7325-4b4b-809a-6d3f002285b4 service cinder] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:54.436022 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7ae7b-7325-4b4b-809a-6d3f002285b4 service cinder] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:54.440740 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:54.441216 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:54.441492 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:54.441764 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] PATH_INFO: `/v3/users/b09153473a954bd5becb7f65a39b8b57` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:54.442882 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b09153473a954bd5becb7f65a39b8b57)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:54.444813 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:54.492875 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b09153473a954bd5becb7f65a39b8b57'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:54.500016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b09153473a954bd5becb7f65a39b8b57'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:54.500636 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b09153473a954bd5becb7f65a39b8b57'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:54.513046 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b09153473a954bd5becb7f65a39b8b57'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:54.513679 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b09153473a954bd5becb7f65a39b8b57'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:54.526065 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1430/2871] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:54 2022] DELETE /identity/v3/users/b09153473a954bd5becb7f65a39b8b57 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:54.530682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4157e2-6604-4478-be63-1e1f7b809cbe None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:54.531174 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4157e2-6604-4478-be63-1e1f7b809cbe None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:54.531294 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4157e2-6604-4478-be63-1e1f7b809cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:54.550362 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-0f4157e2-6604-4478-be63-1e1f7b809cbe None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:54.551491 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0f4157e2-6604-4478-be63-1e1f7b809cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:54.568188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:54.609901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1442/2872] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:54.620199 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:54.621587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:54.678161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114df03e-976a-47ba-8c49-674ca01ed3c6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:54.681901 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9b0dc-3401-4567-96eb-e585fcf544d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:54.682502 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9b0dc-3401-4567-96eb-e585fcf544d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:54.682781 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9b0dc-3401-4567-96eb-e585fcf544d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:54.683052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9b0dc-3401-4567-96eb-e585fcf544d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:54.692615 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9b0dc-3401-4567-96eb-e585fcf544d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:54.693967 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9b0dc-3401-4567-96eb-e585fcf544d4 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:54.703590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1431/2873] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:54.979672 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4157e2-6604-4478-be63-1e1f7b809cbe None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:54.981160 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0f4157e2-6604-4478-be63-1e1f7b809cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:54.994173 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4157e2-6604-4478-be63-1e1f7b809cbe None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:54.998779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:54.999393 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:54.999670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:54.999940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] PATH_INFO: `/v3/projects/1065ceba253f44a888732c412e9f1633` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:55.001115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1065ceba253f44a888732c412e9f1633)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:55.003685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:55.058107 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1065ceba253f44a888732c412e9f1633', 'request_id': 'req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.064531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1065ceba253f44a888732c412e9f1633', 'request_id': 'req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.070842 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1065ceba253f44a888732c412e9f1633', 'request_id': 'req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.084613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] The token cache is being invalidate because project 1065ceba253f44a888732c412e9f1633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:55.084905 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.085617 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.087462 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1443/2874] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:54 2022] DELETE /identity/v3/projects/1065ceba253f44a888732c412e9f1633 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:55.091972 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9b0dc-3401-4567-96eb-e585fcf544d4 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:55.103321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9b0dc-3401-4567-96eb-e585fcf544d4 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:55.111058 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:55.168784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:55.169412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:55.169692 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:55.169994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] PATH_INFO: `/v3/users/ad2be2f880a04b779f1299c9f94059e4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:55.171128 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2be2f880a04b779f1299c9f94059e4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:55.173091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:55.228589 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2be2f880a04b779f1299c9f94059e4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.236134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2be2f880a04b779f1299c9f94059e4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.237540 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2be2f880a04b779f1299c9f94059e4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.250253 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2be2f880a04b779f1299c9f94059e4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.250903 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2be2f880a04b779f1299c9f94059e4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.263630 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1432/2875] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:55 2022] DELETE /identity/v3/users/ad2be2f880a04b779f1299c9f94059e4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:04:55.542315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:55.543909 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:55.561231 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed7b3c-9b52-4ce5-aef3-ed59137b7ff1 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:55.621884 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:55.622506 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:55.622779 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:55.623046 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] PATH_INFO: `/v3/projects/e6a4c0048db24c80a9fda38fe12dd88e` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:55.624178 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a4c0048db24c80a9fda38fe12dd88e)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:55.626055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:55.670270 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:55.671743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a4c0048db24c80a9fda38fe12dd88e', 'request_id': 'req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.679685 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a4c0048db24c80a9fda38fe12dd88e', 'request_id': 'req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.685298 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a4c0048db24c80a9fda38fe12dd88e', 'request_id': 'req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.698116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] The token cache is being invalidate because project e6a4c0048db24c80a9fda38fe12dd88e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:55.698432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.699112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.700613 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1444/2876] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:55 2022] DELETE /identity/v3/projects/e6a4c0048db24c80a9fda38fe12dd88e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:55.705001 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:55.707359 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:55.719981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90383444-4dd1-429e-918b-06d0d6ce23f6 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:55.777159 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:55.777711 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:55.777987 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:55.778255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] PATH_INFO: `/v3/users/afaa1e41df9143858242d6d0500dd639` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:55.779388 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afaa1e41df9143858242d6d0500dd639)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:55.781325 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:55.816125 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:55.830089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afaa1e41df9143858242d6d0500dd639'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.837983 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afaa1e41df9143858242d6d0500dd639'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.838735 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afaa1e41df9143858242d6d0500dd639'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.851450 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afaa1e41df9143858242d6d0500dd639'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.852091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-c6b0f5bf-8d6a-4da5-a406-f374db691bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afaa1e41df9143858242d6d0500dd639'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:55.865787 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1433/2877] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:55 2022] DELETE /identity/v3/users/afaa1e41df9143858242d6d0500dd639 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:56.141202 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.142998 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:56.160598 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeacb51-ed9f-43a1-8d6f-b2f2153beb3f admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.231887 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.232536 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.232820 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.233090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] PATH_INFO: `/v3/projects/160e5ad3d0694a95a2f89a047a65599c` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.234670 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=160e5ad3d0694a95a2f89a047a65599c)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.236895 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:56.302731 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160e5ad3d0694a95a2f89a047a65599c', 'request_id': 'req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:56.310320 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160e5ad3d0694a95a2f89a047a65599c', 'request_id': 'req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:56.316040 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160e5ad3d0694a95a2f89a047a65599c', 'request_id': 'req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:56.329470 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] The token cache is being invalidate because project 160e5ad3d0694a95a2f89a047a65599c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:56.329759 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:56.330369 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-7cedd2b3-8628-4aa7-8ec0-08bbd2d7dca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:56.332604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1445/2878] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:56 2022] DELETE /identity/v3/projects/160e5ad3d0694a95a2f89a047a65599c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:56.337573 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f97b0-3260-4c7f-84c1-241acbb81a45 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.337994 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f97b0-3260-4c7f-84c1-241acbb81a45 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.338399 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f97b0-3260-4c7f-84c1-241acbb81a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.360081 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-cb0f97b0-3260-4c7f-84c1-241acbb81a45 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:56.361444 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-cb0f97b0-3260-4c7f-84c1-241acbb81a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:56.427921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1434/2879] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:56.435963 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045cb7-e6d7-4e98-a337-49c6fcf2d19e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.436377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045cb7-e6d7-4e98-a337-49c6fcf2d19e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.436605 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae045cb7-e6d7-4e98-a337-49c6fcf2d19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.462091 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-ae045cb7-e6d7-4e98-a337-49c6fcf2d19e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:56.463629 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-ae045cb7-e6d7-4e98-a337-49c6fcf2d19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:56.474440 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1446/2880] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:56.478558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f97b0-3260-4c7f-84c1-241acbb81a45 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.488897 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f97b0-3260-4c7f-84c1-241acbb81a45 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.493480 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.494087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.494904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.495178 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.495829 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:56.496161 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.497761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:56.510259 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1435/2881] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:04:56.514611 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045cb7-e6d7-4e98-a337-49c6fcf2d19e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.524657 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae045cb7-e6d7-4e98-a337-49c6fcf2d19e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.529516 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.530112 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.530388 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.530655 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.531286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.533203 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:56.563274 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1447/2882] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:56 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:56.567707 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.577984 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bd40d-ebaa-47a1-8fdd-400d49f5a48d admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.636346 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.637952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19935e4f-9c12-4dd9-95b1-6972ca972854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.638105 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:56.638558 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19935e4f-9c12-4dd9-95b1-6972ca972854 admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.638834 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19935e4f-9c12-4dd9-95b1-6972ca972854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.639101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19935e4f-9c12-4dd9-95b1-6972ca972854 admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.639730 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19935e4f-9c12-4dd9-95b1-6972ca972854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.641668 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19935e4f-9c12-4dd9-95b1-6972ca972854 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:56.668384 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1436/2883] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:56 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:56.669430 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19935e4f-9c12-4dd9-95b1-6972ca972854 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.680702 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19935e4f-9c12-4dd9-95b1-6972ca972854 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.685286 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:56.685421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0c51f-c61d-4e7b-a72b-23528236c724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.685885 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0c51f-c61d-4e7b-a72b-23528236c724 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.686166 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0c51f-c61d-4e7b-a72b-23528236c724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.686440 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0c51f-c61d-4e7b-a72b-23528236c724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.687423 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0c51f-c61d-4e7b-a72b-23528236c724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.689084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0c51f-c61d-4e7b-a72b-23528236c724 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:56.705475 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1437/2884] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:56.706786 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7af00-0797-4a0b-8995-a78c2e585ef5 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.707421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0c51f-c61d-4e7b-a72b-23528236c724 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.707538 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.708115 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.708396 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.708666 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.710431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:56.717346 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0c51f-c61d-4e7b-a72b-23528236c724 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.720139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.721700 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:56.723240 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55c019-fc77-4139-9d8a-b69488b8fa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.723864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55c019-fc77-4139-9d8a-b69488b8fa91 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.724175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55c019-fc77-4139-9d8a-b69488b8fa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.724451 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55c019-fc77-4139-9d8a-b69488b8fa91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.725084 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55c019-fc77-4139-9d8a-b69488b8fa91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.726690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55c019-fc77-4139-9d8a-b69488b8fa91 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:56.730004 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1448/2885] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 11 17:04:56.736775 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1438/2886] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 11 17:04:56.741571 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.753566 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817373e-86b5-4125-99b9-5c17cac996e9 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.758387 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.759064 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.759341 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.759608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] PATH_INFO: `/v3/projects/1e9b74d5d1854020b1eb31f5a4208427/users/f41ecaea3db04a479826a65ff4571119/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.774447 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9b74d5d1854020b1eb31f5a4208427, user_id=f41ecaea3db04a479826a65ff4571119, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.777128 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:56.791110 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1449/2887] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:56 2022] PUT /identity/v3/projects/1e9b74d5d1854020b1eb31f5a4208427/users/f41ecaea3db04a479826a65ff4571119/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:56.805237 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9077f-90e8-47a7-b6f8-365a63a2cd72 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.805523 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9077f-90e8-47a7-b6f8-365a63a2cd72 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.805801 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9077f-90e8-47a7-b6f8-365a63a2cd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.813918 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:56.836673 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-35e9077f-90e8-47a7-b6f8-365a63a2cd72 None None] MFA Rules not processed for user `f41ecaea3db04a479826a65ff4571119`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:56.837942 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-35e9077f-90e8-47a7-b6f8-365a63a2cd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:56.907167 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1439/2888] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:56.919093 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:56.920916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:56.981315 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfa5a9-fb8f-4162-b0d0-becd9d5322c4 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:56.985413 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50d76-1053-4345-a40f-679250a13cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:56.986022 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50d76-1053-4345-a40f-679250a13cbf service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:56.986301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50d76-1053-4345-a40f-679250a13cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:56.986572 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50d76-1053-4345-a40f-679250a13cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:56.995883 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50d76-1053-4345-a40f-679250a13cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:56.997343 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50d76-1053-4345-a40f-679250a13cbf service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:57.007157 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1450/2889] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:57.338467 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9077f-90e8-47a7-b6f8-365a63a2cd72 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:57.340143 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-35e9077f-90e8-47a7-b6f8-365a63a2cd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:57.414831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9077f-90e8-47a7-b6f8-365a63a2cd72 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:57.421016 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:57.421937 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:57.422214 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:57.422564 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] PATH_INFO: `/v3/users/19060feb21d24c369c7f1b806e7457b4` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:57.423894 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=19060feb21d24c369c7f1b806e7457b4)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:57.427724 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:57.480608 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19060feb21d24c369c7f1b806e7457b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:57.488572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19060feb21d24c369c7f1b806e7457b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:57.489238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19060feb21d24c369c7f1b806e7457b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:57.501642 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19060feb21d24c369c7f1b806e7457b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:57.502307 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19060feb21d24c369c7f1b806e7457b4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:57.516717 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1440/2890] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:57 2022] DELETE /identity/v3/users/19060feb21d24c369c7f1b806e7457b4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:57.521877 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712d6d5-49f1-4eb1-8fc4-127cb9b073a7 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:57.522268 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712d6d5-49f1-4eb1-8fc4-127cb9b073a7 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:57.522543 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712d6d5-49f1-4eb1-8fc4-127cb9b073a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:57.542612 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-5712d6d5-49f1-4eb1-8fc4-127cb9b073a7 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:57.543922 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5712d6d5-49f1-4eb1-8fc4-127cb9b073a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:57.608807 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1451/2891] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:57.621690 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:57.623089 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:57.639066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:57.691655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9db54-8fd4-4731-bc4c-b1224e19df2c admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:57.695476 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3512e-261b-4e64-92a4-0e33671aeb92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:57.696137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3512e-261b-4e64-92a4-0e33671aeb92 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:57.696422 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3512e-261b-4e64-92a4-0e33671aeb92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:57.696697 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3512e-261b-4e64-92a4-0e33671aeb92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:57.707330 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3512e-261b-4e64-92a4-0e33671aeb92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:57.708791 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3512e-261b-4e64-92a4-0e33671aeb92 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:57.718449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1441/2892] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:58.006652 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712d6d5-49f1-4eb1-8fc4-127cb9b073a7 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:58.008087 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-5712d6d5-49f1-4eb1-8fc4-127cb9b073a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:58.020208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712d6d5-49f1-4eb1-8fc4-127cb9b073a7 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:58.024821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:58.025449 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.025723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.025993 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] PATH_INFO: `/v3/projects/622a759e4e3b48cb93aa1a6b54b38bed` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.027109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=622a759e4e3b48cb93aa1a6b54b38bed)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:58.029076 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:58.078460 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '622a759e4e3b48cb93aa1a6b54b38bed', 'request_id': 'req-6ed57e42-320e-479e-92c8-83e9868487b6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:58.084628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '622a759e4e3b48cb93aa1a6b54b38bed', 'request_id': 'req-6ed57e42-320e-479e-92c8-83e9868487b6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:58.085826 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:58.091183 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '622a759e4e3b48cb93aa1a6b54b38bed', 'request_id': 'req-6ed57e42-320e-479e-92c8-83e9868487b6'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:58.106075 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] The token cache is being invalidate because project 622a759e4e3b48cb93aa1a6b54b38bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:04:58.106389 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:58.106987 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-6ed57e42-320e-479e-92c8-83e9868487b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:58.108534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1452/2893] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:04:58 2022] DELETE /identity/v3/projects/622a759e4e3b48cb93aa1a6b54b38bed => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:58.114172 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d6d3-535e-4005-be55-a3dc108329ed None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.114509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d6d3-535e-4005-be55-a3dc108329ed None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.115014 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d6d3-535e-4005-be55-a3dc108329ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.136613 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-0228d6d3-535e-4005-be55-a3dc108329ed None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:58.137826 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0228d6d3-535e-4005-be55-a3dc108329ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:58.203401 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1442/2894] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:04:58.212702 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783a239-a134-4c88-985e-88c7a323355e None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.212992 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783a239-a134-4c88-985e-88c7a323355e None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.213269 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783a239-a134-4c88-985e-88c7a323355e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.234954 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-1783a239-a134-4c88-985e-88c7a323355e None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:58.236264 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-1783a239-a134-4c88-985e-88c7a323355e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:58.248921 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1453/2895] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:58.249846 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228d6d3-535e-4005-be55-a3dc108329ed None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:58.259633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228d6d3-535e-4005-be55-a3dc108329ed None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:58.264486 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:58.265181 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.265532 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.265873 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.266653 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:04:58.267031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:58.269294 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:58.290125 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783a239-a134-4c88-985e-88c7a323355e None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:58.291031 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1443/2896] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 11 17:04:58.301036 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783a239-a134-4c88-985e-88c7a323355e None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:58.305892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5468bd8-4381-4b60-916f-c9599e1647f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:58.306593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468bd8-4381-4b60-916f-c9599e1647f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.306938 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468bd8-4381-4b60-916f-c9599e1647f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.307278 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5468bd8-4381-4b60-916f-c9599e1647f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.307981 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5468bd8-4381-4b60-916f-c9599e1647f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:58.309976 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5468bd8-4381-4b60-916f-c9599e1647f6 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:58.344996 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1454/2897] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:04:58 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:58.349470 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:58.367087 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193ea8c-1639-4c5e-ba59-2617e1f60e85 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:58.457623 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29abfb-5ee5-4d4c-b18a-3ea64735c38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:58.458446 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29abfb-5ee5-4d4c-b18a-3ea64735c38f admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.458728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29abfb-5ee5-4d4c-b18a-3ea64735c38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.459085 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29abfb-5ee5-4d4c-b18a-3ea64735c38f admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.459780 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29abfb-5ee5-4d4c-b18a-3ea64735c38f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:58.461877 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29abfb-5ee5-4d4c-b18a-3ea64735c38f admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:58.494741 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1444/2898] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:04:58 2022] POST /identity/v3/users => generated 282 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:04:58.496027 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5468bd8-4381-4b60-916f-c9599e1647f6 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:58.508689 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5468bd8-4381-4b60-916f-c9599e1647f6 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:58.513733 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31f0f7-43fa-4ddf-b1d1-8598d14579cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:58.514338 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f0f7-43fa-4ddf-b1d1-8598d14579cb admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.514764 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f0f7-43fa-4ddf-b1d1-8598d14579cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.514892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31f0f7-43fa-4ddf-b1d1-8598d14579cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.515490 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31f0f7-43fa-4ddf-b1d1-8598d14579cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:58.517103 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31f0f7-43fa-4ddf-b1d1-8598d14579cb admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:58.531335 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1455/2899] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:58.532728 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29abfb-5ee5-4d4c-b18a-3ea64735c38f admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:58.546939 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29abfb-5ee5-4d4c-b18a-3ea64735c38f admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:58.552959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c49f01-f021-4d95-866b-c3eb892b0efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:58.553576 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c49f01-f021-4d95-866b-c3eb892b0efe admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.554302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c49f01-f021-4d95-866b-c3eb892b0efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.554302 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c49f01-f021-4d95-866b-c3eb892b0efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.554771 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c49f01-f021-4d95-866b-c3eb892b0efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:58.556387 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c49f01-f021-4d95-866b-c3eb892b0efe admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:58.568515 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1445/2900] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:04:58.572767 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31f0f7-43fa-4ddf-b1d1-8598d14579cb admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:58.584637 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31f0f7-43fa-4ddf-b1d1-8598d14579cb admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:58.589659 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:58.590258 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.590533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.590801 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] PATH_INFO: `/v3/projects/716f10824a76417b9ea631eb19bf5edc/users/9025e27deb6a46fdb68fbb0297a95f7f/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.609332 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=716f10824a76417b9ea631eb19bf5edc, user_id=9025e27deb6a46fdb68fbb0297a95f7f, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:58.613872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:58.628147 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1456/2901] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:04:58 2022] PUT /identity/v3/projects/716f10824a76417b9ea631eb19bf5edc/users/9025e27deb6a46fdb68fbb0297a95f7f/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:58.638820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09771c-d00b-47cc-8325-7ee9f9ddcb59 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.639182 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09771c-d00b-47cc-8325-7ee9f9ddcb59 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.639460 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09771c-d00b-47cc-8325-7ee9f9ddcb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.670978 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-8e09771c-d00b-47cc-8325-7ee9f9ddcb59 None None] MFA Rules not processed for user `9025e27deb6a46fdb68fbb0297a95f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:58.672825 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8e09771c-d00b-47cc-8325-7ee9f9ddcb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:58.683546 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:58.759881 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1446/2902] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:58.775059 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:58.776984 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:58.858569 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62adbe-c1a5-487b-89f9-bf3603d81dbe admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:58.865177 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba816e-e9ae-4bce-af42-06b3f2728cb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:58.866082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba816e-e9ae-4bce-af42-06b3f2728cb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:58.866535 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba816e-e9ae-4bce-af42-06b3f2728cb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:58.867056 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba816e-e9ae-4bce-af42-06b3f2728cb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:58.878627 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba816e-e9ae-4bce-af42-06b3f2728cb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:58.881581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba816e-e9ae-4bce-af42-06b3f2728cb5 service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:58.891638 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1457/2903] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:04:59.658761 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09771c-d00b-47cc-8325-7ee9f9ddcb59 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:59.661009 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-8e09771c-d00b-47cc-8325-7ee9f9ddcb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:59.732426 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09771c-d00b-47cc-8325-7ee9f9ddcb59 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:59.736909 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:59.738102 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:59.738376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:59.738648 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] PATH_INFO: `/v3/users/f41ecaea3db04a479826a65ff4571119` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:59.739904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f41ecaea3db04a479826a65ff4571119)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:04:59.741866 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:04:59.789331 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f41ecaea3db04a479826a65ff4571119'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:59.797473 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f41ecaea3db04a479826a65ff4571119'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:59.798238 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f41ecaea3db04a479826a65ff4571119'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:59.811995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f41ecaea3db04a479826a65ff4571119'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:59.812790 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f41ecaea3db04a479826a65ff4571119'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:04:59.826042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1447/2904] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:04:59 2022] DELETE /identity/v3/users/f41ecaea3db04a479826a65ff4571119 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:04:59.830633 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2612b-c58a-456b-8add-aafc226ed879 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:59.831349 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2612b-c58a-456b-8add-aafc226ed879 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:59.831788 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2612b-c58a-456b-8add-aafc226ed879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:04:59.851017 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-78a2612b-c58a-456b-8add-aafc226ed879 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:04:59.852276 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-78a2612b-c58a-456b-8add-aafc226ed879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:59.914584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1458/2905] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:04:59.924193 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:04:59.926278 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:04:59.962421 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:04:59.989020 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea43d8-d911-4cd2-a21b-b7fae3377d14 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:04:59.993304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e21da-184d-4255-ac2f-b8030192b325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:04:59.993917 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e21da-184d-4255-ac2f-b8030192b325 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:04:59.994197 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e21da-184d-4255-ac2f-b8030192b325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:04:59.994469 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e21da-184d-4255-ac2f-b8030192b325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.004251 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e21da-184d-4255-ac2f-b8030192b325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:00.005686 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e21da-184d-4255-ac2f-b8030192b325 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:00.015244 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1448/2906] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:05:00.319445 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2612b-c58a-456b-8add-aafc226ed879 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:00.320997 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-78a2612b-c58a-456b-8add-aafc226ed879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:00.334496 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2612b-c58a-456b-8add-aafc226ed879 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:00.339774 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:00.340525 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.340926 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.341296 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] PATH_INFO: `/v3/projects/1e9b74d5d1854020b1eb31f5a4208427` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.342540 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9b74d5d1854020b1eb31f5a4208427)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:00.344584 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:00.368377 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:00.413094 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9b74d5d1854020b1eb31f5a4208427', 'request_id': 'req-73cd6181-03c0-400b-a41b-ede3e32c038e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:00.420201 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9b74d5d1854020b1eb31f5a4208427', 'request_id': 'req-73cd6181-03c0-400b-a41b-ede3e32c038e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:00.427988 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9b74d5d1854020b1eb31f5a4208427', 'request_id': 'req-73cd6181-03c0-400b-a41b-ede3e32c038e'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:00.448530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] The token cache is being invalidate because project 1e9b74d5d1854020b1eb31f5a4208427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:05:00.448879 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:00.449660 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-73cd6181-03c0-400b-a41b-ede3e32c038e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:00.451348 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1459/2907] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:05:00 2022] DELETE /identity/v3/projects/1e9b74d5d1854020b1eb31f5a4208427 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:05:00.457292 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c510ad-9772-4391-9e57-b118bf22dbc9 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.457581 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c510ad-9772-4391-9e57-b118bf22dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.457854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c510ad-9772-4391-9e57-b118bf22dbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.479144 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-f0c510ad-9772-4391-9e57-b118bf22dbc9 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:05:00.480412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-f0c510ad-9772-4391-9e57-b118bf22dbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:00.557719 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1449/2908] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:05:00.569901 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921565-047c-4d3c-9db2-95298a804bf9 None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.570293 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921565-047c-4d3c-9db2-95298a804bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.570723 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921565-047c-4d3c-9db2-95298a804bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.592894 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-66921565-047c-4d3c-9db2-95298a804bf9 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:05:00.594239 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-66921565-047c-4d3c-9db2-95298a804bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:00.603532 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1460/2909] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 11 17:05:00.607572 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c510ad-9772-4391-9e57-b118bf22dbc9 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:00.618092 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c510ad-9772-4391-9e57-b118bf22dbc9 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:00.622853 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:00.623461 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.623734 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.624000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.624680 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81428) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 11 17:05:00.624977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:00.627212 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:00.643848 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1450/2910] 10.222.0.43 () {58 vars in 1153 bytes} [Thu Aug 11 17:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 11 17:05:00.645148 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66921565-047c-4d3c-9db2-95298a804bf9 None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:00.655136 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66921565-047c-4d3c-9db2-95298a804bf9 None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:00.661230 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56538e-e914-42c7-b8e3-9d34f6636651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:00.661936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56538e-e914-42c7-b8e3-9d34f6636651 admin admin] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.662288 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56538e-e914-42c7-b8e3-9d34f6636651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.662630 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56538e-e914-42c7-b8e3-9d34f6636651 admin admin] PATH_INFO: `/v3/projects` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.663673 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56538e-e914-42c7-b8e3-9d34f6636651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:00.666834 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56538e-e914-42c7-b8e3-9d34f6636651 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:00.704872 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1461/2911] 10.222.0.43 () {60 vars in 1153 bytes} [Thu Aug 11 17:05:00 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:05:00.706654 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:00.718856 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eca32-3e3d-48c2-abd9-23c818b9d4cf admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:00.813510 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3988f9-8bc7-45c0-ac67-9836da187b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:00.815231 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3988f9-8bc7-45c0-ac67-9836da187b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.816042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3988f9-8bc7-45c0-ac67-9836da187b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.816380 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3988f9-8bc7-45c0-ac67-9836da187b4e admin admin] PATH_INFO: `/v3/users` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.817158 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3988f9-8bc7-45c0-ac67-9836da187b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:00.820258 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3988f9-8bc7-45c0-ac67-9836da187b4e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:00.866783 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1451/2912] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Aug 11 17:05:00 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 11 17:05:00.871208 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56538e-e914-42c7-b8e3-9d34f6636651 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:00.883082 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56538e-e914-42c7-b8e3-9d34f6636651 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:00.888478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e14fe8-e7da-467e-b9c2-cd115a5bd446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:00.889096 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e14fe8-e7da-467e-b9c2-cd115a5bd446 admin admin] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.889374 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e14fe8-e7da-467e-b9c2-cd115a5bd446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.889648 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e14fe8-e7da-467e-b9c2-cd115a5bd446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.890301 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e14fe8-e7da-467e-b9c2-cd115a5bd446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:00.891965 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e14fe8-e7da-467e-b9c2-cd115a5bd446 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:00.903603 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1462/2913] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:05:00.907731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3988f9-8bc7-45c0-ac67-9836da187b4e admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:00.932248 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3988f9-8bc7-45c0-ac67-9836da187b4e admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:00.937376 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf468f-33f1-4a37-b9dd-c132824d9f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:00.938578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf468f-33f1-4a37-b9dd-c132824d9f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.938578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf468f-33f1-4a37-b9dd-c132824d9f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.938578 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf468f-33f1-4a37-b9dd-c132824d9f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.939708 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf468f-33f1-4a37-b9dd-c132824d9f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:00.941898 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf468f-33f1-4a37-b9dd-c132824d9f12 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:00.953936 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1452/2914] 10.222.0.43 () {58 vars in 1122 bytes} [Thu Aug 11 17:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 11 17:05:00.958220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e14fe8-e7da-467e-b9c2-cd115a5bd446 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:00.970145 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e14fe8-e7da-467e-b9c2-cd115a5bd446 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:00.976309 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:00.976973 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] REQUEST_METHOD: `PUT` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:00.977252 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:00.977529 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] PATH_INFO: `/v3/projects/faa4a53a1a76498a91ffaaf6ec368adf/users/76931f41933948c1ab785aa21a0a26ee/roles/6fb07818f6ff488d9b09556ed4f1d120` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:00.994302 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa4a53a1a76498a91ffaaf6ec368adf, user_id=76931f41933948c1ab785aa21a0a26ee, role_id=6fb07818f6ff488d9b09556ed4f1d120)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:00.996990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:01.012314 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1463/2915] 10.222.0.43 () {60 vars in 1483 bytes} [Thu Aug 11 17:05:00 2022] PUT /identity/v3/projects/faa4a53a1a76498a91ffaaf6ec368adf/users/76931f41933948c1ab785aa21a0a26ee/roles/6fb07818f6ff488d9b09556ed4f1d120 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:05:01.023960 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb2e9c-8032-4ae7-838d-448871f4253f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:01.024281 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb2e9c-8032-4ae7-838d-448871f4253f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:01.024557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb2e9c-8032-4ae7-838d-448871f4253f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:01.056043 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-99bb2e9c-8032-4ae7-838d-448871f4253f None None] MFA Rules not processed for user `76931f41933948c1ab785aa21a0a26ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:05:01.057452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99bb2e9c-8032-4ae7-838d-448871f4253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:01.087587 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:01.126299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1453/2916] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:05:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:05:01.137236 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:01.138940 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:01.201534 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c587a-5a34-4a19-be5b-08378994e900 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:01.206271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca3917-66af-40d4-905c-ddbac098f66a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89757c5dc48942259ffdd67e0cd19ab0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:01.207030 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3917-66af-40d4-905c-ddbac098f66a service cinder] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:01.207339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3917-66af-40d4-905c-ddbac098f66a service cinder] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:01.207616 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3917-66af-40d4-905c-ddbac098f66a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:01.216533 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca3917-66af-40d4-905c-ddbac098f66a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:01.217916 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca3917-66af-40d4-905c-ddbac098f66a service cinder] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:01.227261 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1464/2917] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:05:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:05:11.123455 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=14 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:11.223271 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:17.597312 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb2e9c-8032-4ae7-838d-448871f4253f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:17.598958 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-99bb2e9c-8032-4ae7-838d-448871f4253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:17.661296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb2e9c-8032-4ae7-838d-448871f4253f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:17.664907 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e7a5f-ea0d-4c55-bb77-87c75c922eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:17.665512 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e7a5f-ea0d-4c55-bb77-87c75c922eec service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:17.665789 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e7a5f-ea0d-4c55-bb77-87c75c922eec service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:17.666078 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e7a5f-ea0d-4c55-bb77-87c75c922eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:17.668186 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-963e7a5f-ea0d-4c55-bb77-87c75c922eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:17.733091 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e7a5f-ea0d-4c55-bb77-87c75c922eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:17.734507 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e7a5f-ea0d-4c55-bb77-87c75c922eec service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:17.744831 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1454/2918] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:05:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:05:21.684084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca3917-66af-40d4-905c-ddbac098f66a service cinder] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:21.771741 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca3917-66af-40d4-905c-ddbac098f66a service cinder] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:21.778458 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:21.779337 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:21.779743 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:21.780240 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] PATH_INFO: `/v3/users/76931f41933948c1ab785aa21a0a26ee` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:21.781910 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=76931f41933948c1ab785aa21a0a26ee)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:21.785710 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:21.863109 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76931f41933948c1ab785aa21a0a26ee'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:21.882410 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76931f41933948c1ab785aa21a0a26ee'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:21.883220 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76931f41933948c1ab785aa21a0a26ee'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:21.890539 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76931f41933948c1ab785aa21a0a26ee'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:21.898180 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76931f41933948c1ab785aa21a0a26ee'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:21.903999 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1465/2919] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:05:21 2022] DELETE /identity/v3/users/76931f41933948c1ab785aa21a0a26ee => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:05:21.908452 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa9ee-b15f-47f3-908a-a3ddb3cdc9c5 None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:21.908906 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa9ee-b15f-47f3-908a-a3ddb3cdc9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:21.909000 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2aa9ee-b15f-47f3-908a-a3ddb3cdc9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:21.929498 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-5a2aa9ee-b15f-47f3-908a-a3ddb3cdc9c5 None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:05:21.930704 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5a2aa9ee-b15f-47f3-908a-a3ddb3cdc9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:21.994955 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1455/2920] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:05:22.009749 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:22.012482 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:22.080565 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a541b1-0617-439f-89fb-32066635c73d admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:22.085379 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:22.086090 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:22.086351 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:22.086622 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:22.096026 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:22.098057 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:22.110010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1466/2921] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:05:22.406671 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aa9ee-b15f-47f3-908a-a3ddb3cdc9c5 None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:22.409191 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-5a2aa9ee-b15f-47f3-908a-a3ddb3cdc9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:22.423762 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2aa9ee-b15f-47f3-908a-a3ddb3cdc9c5 None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:22.429420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:22.430066 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:22.430339 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:22.430766 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] PATH_INFO: `/v3/projects/faa4a53a1a76498a91ffaaf6ec368adf` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:22.432069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa4a53a1a76498a91ffaaf6ec368adf)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:22.434256 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:22.482674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:22.486052 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa4a53a1a76498a91ffaaf6ec368adf', 'request_id': 'req-202be032-b023-467c-8eb6-783e621f2ed4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:22.495580 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa4a53a1a76498a91ffaaf6ec368adf', 'request_id': 'req-202be032-b023-467c-8eb6-783e621f2ed4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:22.502137 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa4a53a1a76498a91ffaaf6ec368adf', 'request_id': 'req-202be032-b023-467c-8eb6-783e621f2ed4'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:22.513201 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] The token cache is being invalidate because project faa4a53a1a76498a91ffaaf6ec368adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:05:22.513493 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:22.514254 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-202be032-b023-467c-8eb6-783e621f2ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:22.515794 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1456/2922] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:05:22 2022] DELETE /identity/v3/projects/faa4a53a1a76498a91ffaaf6ec368adf => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:05:32.105409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:32.510080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:40.502695 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:40.504118 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:40.548298 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b3c27-9c24-4f43-a3d1-900309f3d84a service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:40.603229 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:40.603852 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:40.604159 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:40.604431 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] PATH_INFO: `/v3/users/9025e27deb6a46fdb68fbb0297a95f7f` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:40.605588 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9025e27deb6a46fdb68fbb0297a95f7f)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:40.607504 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:40.652682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9025e27deb6a46fdb68fbb0297a95f7f'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:40.664401 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9025e27deb6a46fdb68fbb0297a95f7f'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:40.665055 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9025e27deb6a46fdb68fbb0297a95f7f'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:40.671746 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9025e27deb6a46fdb68fbb0297a95f7f'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:40.679486 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9025e27deb6a46fdb68fbb0297a95f7f'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:40.685446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1467/2923] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:05:40 2022] DELETE /identity/v3/users/9025e27deb6a46fdb68fbb0297a95f7f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:05:40.690080 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660496b9-fb36-4c5e-a2ad-5b1c08be5c0f None None] REQUEST_METHOD: `POST` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:40.690551 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660496b9-fb36-4c5e-a2ad-5b1c08be5c0f None None] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:40.690658 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660496b9-fb36-4c5e-a2ad-5b1c08be5c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:40.718353 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.auth.core [None req-660496b9-fb36-4c5e-a2ad-5b1c08be5c0f None None] MFA Rules not processed for user `ddf38f7f3d7248769f04c04a1b75ce6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81428) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:05:40.719545 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-660496b9-fb36-4c5e-a2ad-5b1c08be5c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:40.778111 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1457/2924] 10.222.0.43 () {58 vars in 958 bytes} [Thu Aug 11 17:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:05:40.787903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:40.789415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:40.846518 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b568e-9004-4c41-9cbb-8f33ec5f6bc2 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:40.850241 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bee826-bc04-4903-9755-d560cfa983be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:40.850837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bee826-bc04-4903-9755-d560cfa983be service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:40.851108 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bee826-bc04-4903-9755-d560cfa983be service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:40.851375 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bee826-bc04-4903-9755-d560cfa983be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:40.857142 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:40.861707 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bee826-bc04-4903-9755-d560cfa983be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:40.863913 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bee826-bc04-4903-9755-d560cfa983be service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:40.874903 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1468/2925] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:05:41.149219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660496b9-fb36-4c5e-a2ad-5b1c08be5c0f None None] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:05:41.150655 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-660496b9-fb36-4c5e-a2ad-5b1c08be5c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:05:41.164067 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660496b9-fb36-4c5e-a2ad-5b1c08be5c0f None None] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:05:41.168727 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:05:41.169368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:05:41.169674 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:05:41.170103 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] PATH_INFO: `/v3/projects/716f10824a76417b9ea631eb19bf5edc` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:05:41.171245 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=716f10824a76417b9ea631eb19bf5edc)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:05:41.173204 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:05:41.219816 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '716f10824a76417b9ea631eb19bf5edc', 'request_id': 'req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:41.227371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '716f10824a76417b9ea631eb19bf5edc', 'request_id': 'req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:41.233666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '716f10824a76417b9ea631eb19bf5edc', 'request_id': 'req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:41.244562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] The token cache is being invalidate because project 716f10824a76417b9ea631eb19bf5edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:05:41.244843 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:41.245462 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:05:41.246921 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1458/2926] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:05:41 2022] DELETE /identity/v3/projects/716f10824a76417b9ea631eb19bf5edc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:05:50.870896 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=3 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:05:51.240989 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:06:07.919899 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41f191-ea0a-4911-b94d-ea9d3acaac0f None None] REQUEST_METHOD: `POST` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:06:07.920400 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41f191-ea0a-4911-b94d-ea9d3acaac0f None None] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:06:07.920483 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41f191-ea0a-4911-b94d-ea9d3acaac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:06:07.940698 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.auth.core [None req-8f41f191-ea0a-4911-b94d-ea9d3acaac0f None None] MFA Rules not processed for user `d7291ccb171e44f89dabc99980312c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 11 17:06:07.941892 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8f41f191-ea0a-4911-b94d-ea9d3acaac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:06:07.996720 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1469/2927] 10.222.0.43 () {58 vars in 967 bytes} [Thu Aug 11 17:06:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 11 17:06:08.037569 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:06:08.039631 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:06:08.108754 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcc9b2-6f73-4a8c-ba0d-138b435605cc admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:06:08.113786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2a9a6-1d9a-4f84-a231-cbcc5983ac89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '256cc27b87684392a22f36429b960644', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:06:08.114415 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2a9a6-1d9a-4f84-a231-cbcc5983ac89 service swift] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:06:08.114718 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2a9a6-1d9a-4f84-a231-cbcc5983ac89 service swift] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:06:08.114986 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2a9a6-1d9a-4f84-a231-cbcc5983ac89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:06:08.123774 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2a9a6-1d9a-4f84-a231-cbcc5983ac89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:06:08.125304 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2a9a6-1d9a-4f84-a231-cbcc5983ac89 service swift] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:06:08.133124 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1459/2928] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Aug 11 17:06:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 11 17:06:14.645455 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41f191-ea0a-4911-b94d-ea9d3acaac0f None None] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:06:14.647432 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-8f41f191-ea0a-4911-b94d-ea9d3acaac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:06:14.718628 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41f191-ea0a-4911-b94d-ea9d3acaac0f None None] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:06:14.723084 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:06:14.723706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:06:14.723986 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:06:14.724292 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] PATH_INFO: `/v3/users/4f62872ebbfd471b9db50bcc401a1768` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:06:14.725663 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f62872ebbfd471b9db50bcc401a1768)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:06:14.727704 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:06:14.771477 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f62872ebbfd471b9db50bcc401a1768'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:14.783897 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f62872ebbfd471b9db50bcc401a1768'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:14.784522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f62872ebbfd471b9db50bcc401a1768'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:14.791361 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f62872ebbfd471b9db50bcc401a1768'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:14.798754 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f62872ebbfd471b9db50bcc401a1768'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:14.804706 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1470/2929] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:06:14 2022] DELETE /identity/v3/users/4f62872ebbfd471b9db50bcc401a1768 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:06:14.984773 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2a9a6-1d9a-4f84-a231-cbcc5983ac89 service swift] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:06:15.001890 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2a9a6-1d9a-4f84-a231-cbcc5983ac89 service swift] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:06:15.064888 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:06:15.065539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:06:15.065820 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:06:15.066090 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] PATH_INFO: `/v3/projects/7cfc50ac33174447af8f4a7ecc6cc125` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:06:15.067932 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cfc50ac33174447af8f4a7ecc6cc125)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:06:15.070120 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:06:15.120854 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cfc50ac33174447af8f4a7ecc6cc125', 'request_id': 'req-43eb5df2-def5-4ba8-9b83-6536d9268596'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.128617 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cfc50ac33174447af8f4a7ecc6cc125', 'request_id': 'req-43eb5df2-def5-4ba8-9b83-6536d9268596'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.136420 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cfc50ac33174447af8f4a7ecc6cc125', 'request_id': 'req-43eb5df2-def5-4ba8-9b83-6536d9268596'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.147879 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] The token cache is being invalidate because project 7cfc50ac33174447af8f4a7ecc6cc125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:06:15.148246 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.148929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.150482 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1460/2930] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:06:14 2022] DELETE /identity/v3/projects/7cfc50ac33174447af8f4a7ecc6cc125 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:06:15.154446 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:06:15.155836 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:06:15.169531 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b8236-c57c-473c-97e1-8fefa6a9c34d admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:06:15.188789 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:06:15.229729 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:06:15.230392 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:06:15.230669 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:06:15.230936 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] PATH_INFO: `/v3/users/46080adfcafc4a239f2772597af6a21d` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:06:15.232088 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46080adfcafc4a239f2772597af6a21d)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:06:15.234011 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:06:15.279773 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46080adfcafc4a239f2772597af6a21d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.292485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46080adfcafc4a239f2772597af6a21d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.293168 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46080adfcafc4a239f2772597af6a21d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.300135 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46080adfcafc4a239f2772597af6a21d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.308409 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46080adfcafc4a239f2772597af6a21d'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.314243 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1471/2931] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:06:15 2022] DELETE /identity/v3/users/46080adfcafc4a239f2772597af6a21d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 11 17:06:15.489368 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:06:15.490908 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:06:15.504612 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb5df2-def5-4ba8-9b83-6536d9268596 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:06:15.525162 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:06:15.566305 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:06:15.566933 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:06:15.567211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:06:15.567490 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] PATH_INFO: `/v3/projects/d4bcfc504d224af9a6fde6c9964dae79` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:06:15.568639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4bcfc504d224af9a6fde6c9964dae79)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:06:15.571069 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:06:15.624280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4bcfc504d224af9a6fde6c9964dae79', 'request_id': 'req-733b1f42-3396-4339-8d1a-55a26ff67b55'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.632509 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4bcfc504d224af9a6fde6c9964dae79', 'request_id': 'req-733b1f42-3396-4339-8d1a-55a26ff67b55'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.638611 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4bcfc504d224af9a6fde6c9964dae79', 'request_id': 'req-733b1f42-3396-4339-8d1a-55a26ff67b55'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.649981 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] The token cache is being invalidate because project d4bcfc504d224af9a6fde6c9964dae79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:06:15.650273 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.650911 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:06:15.652472 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1461/2932] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:06:15 2022] DELETE /identity/v3/projects/d4bcfc504d224af9a6fde6c9964dae79 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:06:25.310821 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:06:25.645864 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:07:18.147944 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:07:18.149478 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:18.209647 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9bb5f-d392-4c00-846e-64d39a573250 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:07:18.258776 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:07:18.259303 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:07:18.259581 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:07:18.259851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:07:18.261604 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:18.332851 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:07:18.334265 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:07:18.345339 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1472/2933] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:07:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:07:19.523283 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:07:19.524784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:19.536959 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b1f42-3396-4339-8d1a-55a26ff67b55 admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:07:19.540562 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:07:19.541262 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:07:19.541539 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:07:19.541807 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:07:19.543468 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:19.561053 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:07:19.562520 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:07:19.574412 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1462/2934] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:07:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:07:28.340522 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=74 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:07:29.568768 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=6 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:07:31.562682 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:07:31.572635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07132bf0-0041-411e-bff1-83591d6bd6b2 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:07:31.576594 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:07:31.577316 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:07:31.577597 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:07:31.577864 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:07:31.579990 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:31.590277 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:07:31.591721 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:07:31.600156 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1473/2935] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:07:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:07:31.712299 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:07:31.713736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:31.724390 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7509e7-ff94-4d4c-b010-1788ad98d30c service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:07:31.729042 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '123d33bb6caa464aba4f9be6b0fb7656', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:07:31.729666 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:07:31.729944 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:07:31.730211 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:07:31.731977 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:31.766784 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:07:31.768234 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:07:31.779952 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1463/2936] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:07:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:07:41.596322 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=6 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:07:41.737932 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:07:41.739321 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:41.775280 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=15 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:07:41.817766 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b37a61-865b-49aa-97e8-514480bfd7f2 service neutron] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:07:41.822949 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:07:41.823551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:07:41.823833 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:07:41.824116 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:07:41.825849 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:07:41.837863 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:07:41.839442 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:07:41.849705 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1474/2937] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:07:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:07:51.845185 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=36 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:08:57.021255 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:08:57.022739 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:08:57.037731 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd8ac0-51d6-4500-bcb8-f300d65fcd80 service neutron] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:08:57.044483 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3819e78802469c83c09a1cdd865f58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:08:57.045096 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:08:57.045371 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:08:57.045639 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:08:57.047448 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:08:57.063633 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:08:57.065321 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:08:57.077716 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1464/2938] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:08:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:09:01.619444 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:09:01.621083 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:09:01.636780 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef0f70-33cc-4c27-b63f-cb2031fd3061 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:09:01.642914 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:09:01.643530 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:09:01.643803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:09:01.644069 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:09:01.646000 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:09:01.661888 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:09:01.663551 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:09:01.674508 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1475/2939] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:09:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:09:05.025681 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:09:05.037904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8c69a-c781-49af-aa50-5c336c46e5cf service nova] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:09:05.048074 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:09:05.048916 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] REQUEST_METHOD: `GET` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:09:05.049363 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:09:05.049699 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:09:05.052059 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:09:05.065147 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:09:05.066751 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:09:05.079893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1465/2940] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:09:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:09:05.094635 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:09:05.096683 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:09:05.164969 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98640f59-96fe-4868-86d1-fc4c3667d632 service placement] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:09:05.169569 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:09:05.170188 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:09:05.170465 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:09:05.170735 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] PATH_INFO: `/v3/users/99baf25bdb4340b8a7947f08e426bd46` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:09:05.171875 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99baf25bdb4340b8a7947f08e426bd46)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:09:05.173803 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:09:05.218726 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99baf25bdb4340b8a7947f08e426bd46'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.230548 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99baf25bdb4340b8a7947f08e426bd46'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.231307 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99baf25bdb4340b8a7947f08e426bd46'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.239845 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99baf25bdb4340b8a7947f08e426bd46'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.247422 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99baf25bdb4340b8a7947f08e426bd46'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.253639 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1476/2941] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:09:05 2022] DELETE /identity/v3/users/99baf25bdb4340b8a7947f08e426bd46 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:09:05.433113 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:09:05.449557 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194e42b-724c-4d54-9d39-dd754d539eeb service placement] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:09:05.507963 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:09:05.508625 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:09:05.508904 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:09:05.509175 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] PATH_INFO: `/v3/projects/e75db9cf63624704afe00066b973360b` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:09:05.510296 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e75db9cf63624704afe00066b973360b)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:09:05.512230 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:09:05.556733 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e75db9cf63624704afe00066b973360b', 'request_id': 'req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.563995 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e75db9cf63624704afe00066b973360b', 'request_id': 'req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.570139 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e75db9cf63624704afe00066b973360b', 'request_id': 'req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.580815 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] The token cache is being invalidate because project e75db9cf63624704afe00066b973360b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:09:05.581101 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.581991 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.583449 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1466/2942] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:09:05 2022] DELETE /identity/v3/projects/e75db9cf63624704afe00066b973360b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:09:05.587485 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:09:05.589086 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:09:05.602459 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910870dd-0913-4721-87e7-7239834d5d13 admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:09:05.658761 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:09:05.659310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:09:05.659586 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:09:05.659853 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] PATH_INFO: `/v3/users/f9a1308590114889884ba944d0b5a105` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:09:05.660995 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9a1308590114889884ba944d0b5a105)` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:09:05.662886 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:09:05.668074 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:09:05.710593 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9a1308590114889884ba944d0b5a105'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.722139 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9a1308590114889884ba944d0b5a105'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.722795 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9a1308590114889884ba944d0b5a105'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.729415 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9a1308590114889884ba944d0b5a105'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.736812 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.notifications [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9a1308590114889884ba944d0b5a105'} {{(pid=81427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:05.745793 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1477/2943] 10.222.0.43 () {58 vars in 1224 bytes} [Thu Aug 11 17:09:05 2022] DELETE /identity/v3/users/f9a1308590114889884ba944d0b5a105 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:09:05.924725 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] Authenticating user token {{(pid=81428) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:09:05.926196 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.fernet_utils [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81428) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:09:05.940893 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a0a86-92d6-4b9d-8d80-2d33154db2ca admin admin] Validating token access rules against request {{(pid=81428) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:09:05.999134 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf38f7f3d7248769f04c04a1b75ce6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '307850a266084dc1bb810ff53de8b0aa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81428) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:09:05.999803 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:09:06.000157 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] SCRIPT_NAME: `/identity` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:09:06.000433 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] PATH_INFO: `/v3/projects/48cc9eed717b48baa3853b5c4cbf33f3` {{(pid=81428) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:09:06.001568 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cc9eed717b48baa3853b5c4cbf33f3)` {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:09:06.003463 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] RBAC: Authorization granted {{(pid=81428) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:09:06.009786 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:09:06.052432 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cc9eed717b48baa3853b5c4cbf33f3', 'request_id': 'req-770f10ab-be1d-47f1-aba6-44387d61753e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:06.059736 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cc9eed717b48baa3853b5c4cbf33f3', 'request_id': 'req-770f10ab-be1d-47f1-aba6-44387d61753e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:06.066590 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cc9eed717b48baa3853b5c4cbf33f3', 'request_id': 'req-770f10ab-be1d-47f1-aba6-44387d61753e'} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:06.076929 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] The token cache is being invalidate because project 48cc9eed717b48baa3853b5c4cbf33f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81428) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 11 17:09:06.077219 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:06.077851 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG keystone.notifications [None req-770f10ab-be1d-47f1-aba6-44387d61753e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81428) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 11 17:09:06.079345 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: [pid: 81428|app: 0|req: 1467/2944] 10.222.0.43 () {58 vars in 1233 bytes} [Thu Aug 11 17:09:05 2022] DELETE /identity/v3/projects/48cc9eed717b48baa3853b5c4cbf33f3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 11 17:09:15.739837 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:09:16.073842 dv-cinder-852208-4-iscsi devstack@keystone.service[81428]: DEBUG dbcounter [-] [81428] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81428) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 11 17:12:46.722580 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] Authenticating user token {{(pid=81427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 11 17:12:46.724058 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:12:46.823585 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38550472-567d-4be6-8b4a-1d6d64e2c0bf admin admin] Validating token access rules against request {{(pid=81427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 11 17:12:46.885310 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd125d3-2c3b-4570-affc-3e778b57dbd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08abc209f6cd4b62b34cad53c8ba4c54', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d968a7efd10472db2d392353d807180', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 11 17:12:46.885923 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd125d3-2c3b-4570-affc-3e778b57dbd8 service placement] REQUEST_METHOD: `GET` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 11 17:12:46.886199 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd125d3-2c3b-4570-affc-3e778b57dbd8 service placement] SCRIPT_NAME: `/identity` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 11 17:12:46.886468 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd125d3-2c3b-4570-affc-3e778b57dbd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 11 17:12:46.888227 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.fernet_utils [None req-abd125d3-2c3b-4570-affc-3e778b57dbd8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 11 17:12:46.900010 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd125d3-2c3b-4570-affc-3e778b57dbd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 11 17:12:46.901631 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd125d3-2c3b-4570-affc-3e778b57dbd8 service placement] RBAC: Authorization granted {{(pid=81427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 11 17:12:46.912608 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: [pid: 81427|app: 0|req: 1478/2945] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Aug 11 17:12:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 11 17:12:56.907245 dv-cinder-852208-4-iscsi devstack@keystone.service[81427]: DEBUG dbcounter [-] [81427] Writing DB stats keystone:SELECT=72 {{(pid=81427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}